CI Execution

Use the twd-cli package to run TWD tests in headless CI environments. It wraps Puppeteer, waits for your app, executes all tests, and reports coverage. It exits with a non-zero status code when a test fails, so it integrates directly into any CI/CD pipeline.

You can find the source code, release notes, and issue tracker at github.com/BRIKEV/twd-cli.

Install

npm install twd-cli

or run it directly:

npx twd-cli run

How It Works

Puppeteer is not used as a testing framework — it simply provides a headless browser to load your application. Once the page loads, all test execution happens inside the real browser context through the TWD runner.

1. Launches a headless browser via Puppeteer
2. Navigates to your dev server URL
3. Waits for the app and TWD sidebar to be ready
4. TWD's in-browser test runner executes all tests against the real DOM
5. Collects and reports test results
6. Validates collected mocks against OpenAPI contracts (if configured)
7. Optionally collects code coverage data
8. Exits with appropriate code (0 for success, 1 for failures)

Configure (optional)

Create twd.config.json in your repo to customize the runner:

{
  "url": "http://localhost:5173",
  "timeout": 10000,
  "coverage": true,
  "coverageDir": "./coverage",
  "nycOutputDir": "./.nyc_output",
  "headless": true,
  "puppeteerArgs": ["--no-sandbox", "--disable-setuid-sandbox"],
  "retryCount": 2,
  "protocolTimeout": 300000,
  "maxFailures": 10,
  "chunkSize": 10,
  "contracts": [],
  "contractReportPath": ".twd/contract-report.md"
}

Option	Type	Default	Description
url	string	"http://localhost:5173"	Dev server URL to open before running tests
timeout	number	10000	Milliseconds to wait for the page/sidebar
coverage	boolean	true	Toggle code coverage collection
coverageDir	string	"./coverage"	Output folder for coverage reports
nycOutputDir	string	"./.nyc_output"	NYC temp folder
headless	boolean	true	Run Chrome in headless mode
puppeteerArgs	string[]	["--no-sandbox", "--disable-setuid-sandbox"]	Extra arguments for Puppeteer
retryCount	number	2	Number of times to attempt each test before reporting failure. Default is 2 (one normal attempt + one retry). Set to 1 to disable retries.
protocolTimeout	number	300000	Puppeteer CDP protocolTimeout in ms (5 min). Tests run in chunks, so this bounds a single chunk's browser call, not the entire run. Raise it (e.g. 600000) for slow CI or if individual chunks hang. 0 means no timeout.
maxFailures	number	10	Stop the run once this many tests have failed in total. The CLI prints the results gathered so far and exits non-zero. Set 0 to disable and always run every test. Note this limit is per shard when sharding.
chunkSize	number	10	How many tests run per browser call. Smaller values make the failure limit and timeouts more granular (less work lost if one chunk hangs), larger values reduce overhead. 0 runs everything in one call.
contracts	object[]	[]	OpenAPI contract validation specs. See Contract Testing
contractReportPath	string	—	Path to write a markdown report for CI/PR integration
record	object	see Recording Runs	Video recording settings

Filtering tests

Run only a subset of tests with the repeatable --test flag. Matching is case-insensitive and matches a substring of each test's full "Suite > test name" path:

# Every test whose name contains "shows error"
npx twd-cli run --test "shows error"

# Because matching uses the full "suite > test" path, passing a describe name
# runs every test inside that describe block:
npx twd-cli run --test "Login"

# Multiple --test flags are combined with OR (a test runs if it matches any):
npx twd-cli run --test "Login" --test "Signup"

Two things to know:

• If no test matches any filter, the run exits with code 1 and prints No tests matched filter(s): ..., so a typo will not silently look like a pass.
• Code coverage collection is skipped while a --test filter is active, since a filtered run is a partial (debug) run.

--test and --shard compose. Filters resolve first, then the filtered list is sharded. See Sharding.

TIP

twd-relay accepts the same --test flag for driving a run from an AI agent. See AI Remote Testing.

GitHub Action (Recommended)

The easiest way to run TWD tests in CI. The composite action handles Puppeteer caching, Chrome installation, and optional contract report posting in a single step:

name: TWD Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

permissions:
  pull-requests: write  # only needed if using contract-report

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v5

      - uses: actions/setup-node@v5
        with:
          node-version: 24
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Install mock service worker
        run: npx twd-js init public --save

      - name: Start dev server
        run: |
          nohup npm run dev > /dev/null 2>&1 &
          npx wait-on http://localhost:5173

      - name: Run TWD tests
        uses: BRIKEV/twd-cli/.github/actions/run@main
        with:
          contract-report: 'true'

Action Inputs

Input	Default	Description
working-directory	.	Directory where twd.config.json lives
contract-report	false	Post contract validation summary as a PR comment
shard	(empty)	Run one shard of the suite, as <index>/<total> (e.g. 2/4). Leave empty to run everything in one job. See Sharding
report-dir	.twd/run	Where the shard report is written. Only used when shard is set
upload-report	true	Upload the shard report as an artifact named twd-run-<index>, the layout twd-cli merge expects. Only used when shard is set

With code coverage

The action runs in the same job, so coverage data is available for subsequent steps:

      - name: Run TWD tests
        uses: BRIKEV/twd-cli/.github/actions/run@main

      - name: Display coverage
        run: npm run collect:coverage:text

Custom Setup (Without the Action)

If you prefer full control over each CI step, or your CI isn't GitHub Actions, set up each step manually. Puppeteer 24+ no longer auto-downloads Chrome, so you need to install it explicitly:

- name: Install dependencies
  run: npm ci

- name: Install mock service worker
  run: npx twd-js init public --save

- name: Cache Puppeteer browsers
  uses: actions/cache@v4
  with:
    path: ~/.cache/puppeteer
    key: ${{ runner.os }}-puppeteer-${{ hashFiles('package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-puppeteer-

- name: Install Chrome for Puppeteer
  run: npx puppeteer browsers install chrome

- name: Run TWD tests
  run: npx twd-cli run

Tip: Puppeteer 24+ no longer downloads Chrome automatically. Either run npx puppeteer browsers install chrome in CI or cache ~/.cache/puppeteer between runs to avoid repeated downloads.

Custom Runner Options

If you're building your own CI script instead of using twd-cli, you can pass options to the TestRunner constructor to handle flaky CI environments:

const runner = new TestRunner({
  onStart: () => {},
  onPass: (test, retryAttempt) => {
    const suffix = retryAttempt ? ` (retry ${retryAttempt}/2)` : '';
    testStatus.push({ id: test.id, status: "pass" });
    console.log(`✓ ${test.name}${suffix}`);
  },
  onFail: (test, err) => {
    testStatus.push({ id: test.id, status: "fail", error: err.message });
  },
  onSkip: (test) => {
    testStatus.push({ id: test.id, status: "skip" });
  },
}, { retryCount: 2 });

Option	Type	Default	Description
retryCount	number	1	Total number of attempts per test. 1 means no retry. 2 means one original attempt + one retry on failure.

The onPass callback receives an optional second parameter retryAttempt — it is undefined when the test passes on the first attempt, or the attempt number (2+) when it passes on a retry. This lets you log which tests are flaky so you can fix them later.

Note: The retry mechanism re-runs the full test cycle (beforeEach hooks → test → afterEach hooks) on each attempt, ensuring clean state between retries.

Cross-browser testing (experimental)

Experimental

For the vast majority of projects, twd-cli is the recommended runner and covers ~90% of use cases — it's faster, collects coverage, validates contracts, and is battle-tested. twd-runner is an experimental complement; reach for it only when you specifically need to validate other browser engines.

twd-cli runs your tests in headless Chromium (via Puppeteer). If you also want to catch Firefox and WebKit (Safari) engine differences, twd-runner runs the same TWD tests across engines using Playwright. It's a complement to twd-cli, not a replacement: keep twd-cli as your primary runner (coverage + contracts), and add twd-runner as an extra cross-browser check.

npm install -D twd-runner
npx playwright install   # downloads the browser binaries (npm install does not)
npx twd-runner run

It reads the same twd.config.json. The keys that matter most here:

Option	Default	Description
browsers	["chromium","firefox","webkit"]	Engines to run, in parallel within a single job
waitForServiceWorker	false	Set true for apps that mock via a service worker. Firefox/WebKit can be slow to take control of the page, and mocks registered before then are silently dropped. Enabling it also auto-warms the dev server first (a cold dev server otherwise races SW registration).

A single job runs the engines in parallel, so the same two steps work on any CI — no GitHub-specific matrix required:

- name: Start dev server
  run: |
    nohup npm run dev > /dev/null 2>&1 &
    npx wait-on http://localhost:5173

- name: Run cross-browser tests
  run: npx twd-runner run

Recommended split

Run your full suite on Chromium with twd-cli (coverage, contracts, retries), and use twd-runner only for a cross-browser pass on the engines Puppeteer can't reach — typically "browsers": ["firefox", "webkit"]. twd-runner does not collect coverage or run contract validation, and is slower than twd-cli.

Next Steps

• Sharding: Split a long run across parallel CI jobs and merge the reports (beta)
• Recording Runs: Record a run to video, paced so it is watchable in a pull request
• Contract Testing: Validate your API mocks against OpenAPI specs
• Code Coverage: Learn how to collect and report code coverage with TWD
• Writing Tests: Create testable components
• API Mocking: Test with network requests
• API Reference: Complete function documentation