QA workspace, previews, screenshots, and launch gating

Simple terms

QA is the safety check before traffic. It proves the variant renders, tracking works, and blockers are resolved.

QA proves that a variant renders correctly, tracking fires, screenshots are captured, and launch blockers are resolved before visitors see a test.

Routes

• /tests/[id]?tab=qa
• /tests/[id]/variations/[varId]

Implementation behavior

• Installed stores can use Edit on store as the high-fidelity draft surface. The editor runs on the real merchant origin and saves into the same variation artifacts used by the dashboard.
• Quick preview remains the zero-install proxy fallback. Cart, checkout, consent, authentication, and other session-dependent behavior can differ from the live store and should be verified on-site before launch.
• Preview sessions force a specific experiment and variation through cookies or preview parameters.
• QA run APIs inspect the experiment setup, target page, goals, screenshots, and runtime behavior.
• Screenshot jobs can capture desktop, iPhone, and iPad views and persist result URLs.
• Goal verification checks whether the selected metric can be observed or triggered in the preview path.
• Launch gates combine setup blockers, QA state, guardrails, and experiment status.

How to use it

• Open QA from an experiment before pressing start.
• Preview every variation on the intended URL and key device sizes.
• Run screenshots and goal verification for the primary metric.
• Fix every blocker or consciously document the exception before launch.

When not to use it

• Do not use QA as a substitute for setting up the experiment correctly.
• Do not approve a test from desktop screenshots only when mobile traffic matters.
• Do not ignore a failed goal verification just because the visual change looks right.

Implementation source

• The QA tab is the ?tab=qa view of src/app/(dashboard)/tests/[id]/page.tsx, with deeper runs inside the variation editor.
• Handlers include /api/experiments/[id]/qa/run, /api/experiments/[id]/qa/screenshots, /api/experiments/[id]/qa/goal-verification, /api/preview, and /api/preview-sessions.
• Screenshot jobs and preview sessions persist evidence used by launch gates.

Data and API

• Prisma ScreenshotJob, preview session records, operator evidence, goal verification records, and experiment QA state.
• APIs: /api/experiments/[id]/qa/run, /api/experiments/[id]/qa/goal-verification, /api/experiments/[id]/qa/screenshots, /api/preview, /api/preview-sessions.
• Public screenshot readback exists under /api/v1/screenshots/[jobId].

Failure modes

• Preview shows control: force cookie/session missing or targeting does not match.
• Edit on store does not connect: confirm the SDK is installed on the target URL and that its origin belongs to the active shop.
• Screenshot job stuck: check browser worker availability and target page accessibility.
• Goal verification false negative: selector may need user interaction, delayed render, or a different URL state.