Experiment E2E Generator

Appearance

Test Patterns ​

This page catalogs common patterns used in A/B experiment E2E tests. Each pattern includes a description, when to use it, and a generic code example.

Live URL Testing ​

Navigate to QA URLs with Adobe Target preview tokens (or similar A/B platforms) and verify the DOM matches the expected variant.

When to use: Testing experiments deployed to a QA environment with variant switching via URL parameters or cookies.

js
test.describe('Experiment - Live URL', () => {
  test('should show experiment variant via preview token', async ({ page }) => {
    const previewUrl = 'https://www.example.com/nl/product/?at_preview_token=abc123';
    await page.goto(previewUrl);
    await page.waitForLoadState('networkidle');

    const experimentComponent = page.locator('[data-experiment="my-experiment"]');
    await expect(experimentComponent).toBeVisible();
  });
});

Bundle Injection Testing ​

Load the built experiment bundle (dist/v1-index.jsx) into a blank page or live page via page.addScriptTag(), then verify the experiment modifies the DOM correctly.

When to use: Smoke-testing the experiment bundle independently, without relying on the full QA environment or A/B platform.

js
import { readFileSync, existsSync } from 'fs';
import { join } from 'path';

const BUNDLE_PATH = join(process.cwd(), 'dist', 'v1-index.jsx');

function loadBundle() {
  if (!existsSync(BUNDLE_PATH)) {
    throw new Error(`Bundle not found. Run \`yarn build\` first.`);
  }
  return readFileSync(BUNDLE_PATH, 'utf8');
}

test.describe('Bundle Injection', () => {
  let bundleCode;

  test.beforeAll(() => {
    bundleCode = loadBundle();
  });

  test.beforeEach(async ({ page }) => {
    await page.goto('about:blank');
    await page.addScriptTag({ content: bundleCode });
  });

  test('should render experiment component', async ({ page }) => {
    const component = page.locator('.experiment-component');
    await component.waitFor({ state: 'visible', timeout: 10000 });
    await expect(component).toBeVisible();
  });
});

Pre-Validation Pattern ​

Check that the target element exists on the page before running experiment-specific assertions. This prevents false negatives when the page structure changes.

When to use: When your experiment attaches to an existing DOM element that might not always be present (e.g., a product page section).

js
test('should modify target element when it exists', async ({ page }) => {
  await page.goto(experimentUrl);

  // Pre-validate: check the target element exists
  const targetSection = page.locator('#product-details');
  const targetExists = await targetSection.isVisible();

  if (!targetExists) {
    test.skip('Target element not present on this page');
    return;
  }

  // Now verify the experiment modification
  const experimentBadge = targetSection.locator('.experiment-badge');
  await expect(experimentBadge).toBeVisible();
});

Multi-Market Iteration ​

Loop over markets and page paths for combinatorial coverage. This pattern tests the same assertions across every configured market.

When to use: When your experiment runs across multiple markets and you need to verify it works in each one.

js
import { experimentConfig, qaLinksConfig } from '../../config/index.js';

const pagePaths = ['/smartphones/', '/tablets/', '/accessories/'];

for (const market of experimentConfig.markets) {
  for (const pagePath of pagePaths) {
    test.describe(`${market.name} - ${pagePath}`, () => {
      test('should display experiment banner', async ({ page }) => {
        const { experimentUrl } = qaLinksConfig.getUrls(market.code);
        await page.goto(`${experimentUrl}${pagePath}`);

        const banner = page.locator('[data-experiment="promo-banner"]');
        await expect(banner).toBeVisible();
      });
    });
  }
}

Set consent or tracking cookies before navigating to the test page. Many sites require cookie consent before displaying experiment content.

When to use: When the experiment is gated behind cookie consent, or you need specific cookies for the A/B platform to activate.

js
test.describe('With Cookie Consent', () => {
  test.beforeEach(async ({ context, page }) => {
    // Set cookie consent before navigating
    await context.addCookies([
      {
        name: 'cookie_consent',
        value: 'accepted',
        domain: '.example.com',
        path: '/',
      },
      {
        name: 'ab_platform_id',
        value: 'test-user-123',
        domain: '.example.com',
        path: '/',
      },
    ]);

    await page.goto(experimentUrl);
  });

  test('should show experiment after consent', async ({ page }) => {
    const experimentWidget = page.locator('.experiment-widget');
    await expect(experimentWidget).toBeVisible();
  });
});

Soft Assertions Pattern ​

Use expect.soft() for non-blocking checks, allowing a test to continue and collect all failures instead of stopping at the first one. Capture a screenshot at the end for debugging.

When to use: Multi-page sweeps or visual checks where you want a full picture of failures, not just the first one.

js
test('should display correct content across all sections', async ({ page }) => {
  await page.goto(experimentUrl);

  // Soft assertions - test continues even if one fails
  await expect.soft(page.locator('.hero-banner')).toBeVisible();
  await expect.soft(page.locator('.hero-banner h1')).toHaveText('New Collection');
  await expect.soft(page.locator('.promo-section')).toBeVisible();
  await expect.soft(page.locator('.promo-section .price')).toContainText('$');

  // Screenshot at the end captures the state for all assertions
  await page.screenshot({ path: 'screenshots/full-page-check.png', fullPage: true });
});

Clipboard Testing Pattern ​

Verify copy-to-clipboard functionality by reading the clipboard contents after a user interaction.

When to use: When your experiment includes a "copy code" or "share link" feature.

js
// Grant clipboard permissions in playwright.config.js:
// use: { permissions: ['clipboard-read', 'clipboard-write'] }

test('should copy promo code to clipboard', async ({ page, context }) => {
  await context.grantPermissions(['clipboard-read', 'clipboard-write']);
  await page.goto(experimentUrl);

  const copyButton = page.getByRole('button', { name: 'Copy Code' });
  await copyButton.click();

  // Read clipboard contents
  const clipboardText = await page.evaluate(() => navigator.clipboard.readText());
  expect(clipboardText).toBe('SAVE20');
});

Viewport Testing Pattern ​

Test how the experiment renders across different viewport sizes.

When to use: When your experiment has responsive behavior or mobile-specific variants.

js
const viewports = [
  { name: 'mobile', width: 375, height: 667 },
  { name: 'tablet', width: 768, height: 1024 },
  { name: 'desktop', width: 1440, height: 900 },
];

for (const viewport of viewports) {
  test(`should render correctly on ${viewport.name}`, async ({ page }) => {
    await page.setViewportSize({ width: viewport.width, height: viewport.height });
    await page.goto(experimentUrl);

    const banner = page.locator('.experiment-banner');
    await expect(banner).toBeVisible();

    await page.screenshot({
      path: `screenshots/experiment-${viewport.name}.png`,
    });
  });
}

What's Next ​