Run safe commerce experiments

Experiment Lab lets tenant managers test guidance before making it a standard part of support. Supported tests include reply guidance, recommendations, procedure copy, and macro copy.

Build and launch

  1. State a measurable hypothesis.
  2. Enter the current control and proposed variant.
  3. Select a primary metric and allocate 1% to 50% of eligible conversations to the variant.
  4. Add any extra content guardrails and save the draft.
  5. Start the version, then refresh the safe cohort to make stable, deterministic assignments.
  6. Record success, not successful, or excluded after each outcome is known.

Versioning and rollback

Create the next version instead of overwriting an existing test. Every version retains its control, variant, hypothesis, allocation, assignments, and outcomes. Pause assignments at any time, complete the test, or roll it back immediately.

Experiments change guidance only. They cannot broaden Shopify permissions, refund or cancellation authority, approval thresholds, eligibility rules, or other safety policy. Refreshing a cohort does not send messages or execute commerce actions.