Best Practices

This page covers locator strategy, assertion patterns, and stability tips for writing reliable A/B experiment tests.

Locator Strategy

Use locators in this priority order:

1. Role-Based Locators (Preferred)

The most resilient locators. They match accessible roles and are resistant to CSS/HTML changes:

// Buttons
page.getByRole('button', { name: 'Add to Cart' })

// Links
page.getByRole('link', { name: 'View Details' })

// Headings
page.getByRole('heading', { name: 'Special Offer' })

// Images
page.getByRole('img', { name: 'Product photo' })

2. Semantic Locators

Use when role-based locators aren't specific enough:

// Text content
page.getByText('Free shipping on orders over $50')

// Labels (form fields)
page.getByLabel('Promo Code')

// Placeholders
page.getByPlaceholder('Enter your code')

// Alt text
page.getByAltText('Experiment banner image')

3. Data Attributes

Use for experiment-specific elements that lack semantic meaning:

// Data attribute selectors
page.locator('[data-experiment="my-experiment"]')
page.locator('[data-testid="promo-banner"]')

4. CSS Selectors (Last Resort)

Avoid when possible. CSS selectors break when class names or structure changes:

// Fragile - avoid
page.locator('.promo-banner__container > .inner-wrapper h2')

// Slightly better - scoped and simple
page.locator('.experiment-component')

Web-First Assertions

Playwright's web-first assertions automatically wait and retry. Always prefer these over manual checks.

Visibility

// Preferred - auto-waits up to the assertion timeout
await expect(page.locator('.banner')).toBeVisible();
await expect(page.locator('.banner')).not.toBeVisible();

// Avoid - manual wait + boolean check
const isVisible = await page.locator('.banner').isVisible(); // No auto-retry

Text Content

// Exact match
await expect(page.locator('.title')).toHaveText('Welcome');

// Partial match
await expect(page.locator('.description')).toContainText('limited time');

// Regex match
await expect(page.locator('.price')).toHaveText(/\$\d+\.\d{2}/);

CSS Properties

await expect(page.locator('.cta')).toHaveCSS('background-color', 'rgb(0, 119, 200)');
await expect(page.locator('.overlay')).toHaveCSS('display', 'block');

URL and Page Title

await expect(page).toHaveURL(/\/checkout\//);
await expect(page).toHaveTitle('Shopping Cart');

Count

await expect(page.locator('.product-card')).toHaveCount(4);

Config-Driven Testing

Keep expected values in config files rather than hardcoding them in tests. This makes multi-market testing manageable.

// tests/config/content.config.js
export const expectedContent = {
  NL: { headline: 'Speciale Aanbieding', cta: 'Nu Kopen' },
  BE: { headline: 'Offre Speciale', cta: 'Acheter Maintenant' },
};

// In your test
import { expectedContent } from '../../config/content.config.js';

for (const [marketCode, content] of Object.entries(expectedContent)) {
  test(`should show correct text in ${marketCode}`, async ({ page }) => {
    await page.goto(getMarketUrl(marketCode));
    await expect(page.locator('.headline')).toHaveText(content.headline);
    await expect(page.getByRole('button')).toHaveText(content.cta);
  });
}

Soft Assertions

Use expect.soft() for multi-check sweeps where you want all failures reported, not just the first:

test('should display all experiment elements', async ({ page }) => {
  await page.goto(experimentUrl);

  await expect.soft(page.locator('.hero')).toBeVisible();
  await expect.soft(page.locator('.hero h1')).toHaveText('New Arrivals');
  await expect.soft(page.locator('.promo-code')).toBeVisible();
  await expect.soft(page.locator('.countdown')).toBeVisible();

  // Capture state for debugging all failures
  await page.screenshot({ path: 'screenshots/all-elements-check.png', fullPage: true });
});

Screenshot on Failure

The generated config takes screenshots automatically (screenshot: 'on'). For additional debugging screenshots:

test('should render banner', async ({ page }) => {
  await page.goto(experimentUrl);

  try {
    await expect(page.locator('.banner')).toBeVisible({ timeout: 5000 });
  } catch {
    // Extra screenshot for debugging
    await page.screenshot({ path: 'screenshots/banner-not-found.png', fullPage: true });
    throw new Error('Banner not visible - see screenshot');
  }
});

Scroll Into View

Elements below the fold may not be interactive until scrolled into view:

import { scrollIntoView } from '../utils/test-helpers.js';

test('should show footer experiment', async ({ page }) => {
  await page.goto(experimentUrl);

  const footerBanner = page.locator('.footer-experiment');

  // Scroll the element into view first
  await scrollIntoView(footerBanner);
  await expect(footerBanner).toBeVisible();
});

Wait for Page Ready

Before asserting, ensure the page is fully loaded:

test('should show experiment after page load', async ({ page }) => {
  await page.goto(experimentUrl);

  // Wait for network to be idle (no pending requests)
  await page.waitForLoadState('networkidle');

  // Or wait for DOM content loaded
  await page.waitForLoadState('domcontentloaded');

  // Now assert
  await expect(page.locator('.experiment')).toBeVisible();
});

Test Independence

Each test should be self-contained. Never rely on state from a previous test:

// Bad - tests depend on each other
test('step 1: open modal', async ({ page }) => {
  await page.getByRole('button', { name: 'Open' }).click();
});

test('step 2: verify modal content', async ({ page }) => {
  // This fails because the modal isn't open - different browser context
  await expect(page.locator('.modal')).toBeVisible();
});

// Good - each test sets up its own state
test('should display modal content when opened', async ({ page }) => {
  await page.goto(experimentUrl);
  await page.getByRole('button', { name: 'Open' }).click();
  await expect(page.locator('.modal')).toBeVisible();
  await expect(page.locator('.modal')).toContainText('Welcome');
});

Narrow Selector Scope

Scope selectors to the experiment component to avoid false positives from other page content:

// Bad - matches any heading on the page
await expect(page.getByRole('heading', { name: 'Shop Now' })).toBeVisible();

// Good - scoped to the experiment component
const experiment = page.locator('[data-experiment="my-experiment"]');
await expect(experiment.getByRole('heading', { name: 'Shop Now' })).toBeVisible();

Avoid Hardcoded Waits

Use Playwright's built-in waiting instead of setTimeout:

// Bad - arbitrary delay
await new Promise(resolve => setTimeout(resolve, 3000));
await expect(page.locator('.banner')).toBeVisible();

// Good - auto-wait with timeout
await expect(page.locator('.banner')).toBeVisible({ timeout: 10000 });

// Good - wait for specific condition
await page.waitForLoadState('networkidle');
await page.locator('.spinner').waitFor({ state: 'hidden' });

Summary Checklist

Before committing your tests, verify:

• [ ] Locators use getByRole where possible
• [ ] Assertions use web-first matchers (toBeVisible, toHaveText)
• [ ] Expected values come from config, not hardcoded strings
• [ ] Tests are independent (no shared state)
• [ ] Selectors are scoped to the experiment component
• [ ] No hardcoded setTimeout waits
• [ ] Screenshots capture failure state for debugging
• [ ] Tests run successfully in all configured browsers

What's Next

• Writing Tests - step-by-step test writing tutorial
• Test Patterns - common A/B test patterns
• Running Tests - execution, debugging, CI