Skip to main content
Ask AI

Package Overview

testrelic-playwright is a single pip package that bundles a pytest plugin and a companion command-line tool. This page explains the moving parts so you know what each one does before you configure them.

The pytest-plugin model​

The plugin registers through a pytest11 entry point. As soon as testrelic-playwright is installed in the same environment as pytest, it activates automatically — there is nothing to import and no conftest.py wiring.

During a run it observes Playwright's page and APIRequestContext fixtures, the pytest step protocol, and expect() assertions, then assembles an analytics timeline. Navigation is tracked automatically, including SPA / client-side route changes that don't trigger a full page load. At the end of the run it writes the JSON and HTML reports and, if configured, uploads to the cloud.

This matches the JS @testrelic/playwright-analytics reporter feature-for-feature: expect()-assertion tracking, action steps, console logs, screenshot and video artifacts, SPA navigation detection, APIRequestContext wrapping with URL filtering, the interactive HTML report and local report server, large-suite streaming, and shard merge.

The testrelic_options mechanism​

All behavior is controlled through a single testrelic_options list, set in your project's pytest config. You can use either pyproject.toml or pytest.ini:

pyproject.toml
[tool.pytest.ini_options]
testrelic_options = [
  "includeStackTrace = true",
  "codeContextLines = 5",
]
pytest.ini
[pytest]
testrelic_options =
  includeStackTrace = true
  codeContextLines = 5

A few common toggles are also exposed as CLI flags (--testrelic-output, --testrelic-disable, --testrelic-quiet). See Configuration for the full reference.

The testrelic-playwright CLI​

The package installs a testrelic-playwright command for working with reports outside of a test run:

CommandPurpose
merge shard-1.json shard-2.json -o merged.jsonMerge timeline JSON from multiple shards into one report
serve ./test-results/.testrelic-report --port 9323 [--host 127.0.0.1] [--open]Serve a streaming report directory locally
open ./test-results/.testrelic-report [--port 9323]Open a report directory in the browser
drain [--queue-dir PATH]Flush the offline upload queue to the cloud
versionPrint the installed package version

Streaming pipeline for large suites​

For most suites the report is generated embedded — a single self-contained HTML file. For very large suites the plugin can switch to streaming mode, where timeline data is streamed to disk instead of held in memory. This is governed by two options:

  • reportMode — embedded, streaming, or auto (the default).
  • streamingThreshold — the test count at which auto switches to streaming (default 500).

A streamed report is written to a .testrelic-report directory rather than a single file.

Local report server​

Streamed reports are directory-based, so you view them through the bundled server rather than by opening a file directly:

Terminal
testrelic-playwright serve ./test-results/.testrelic-report --port 9323 --open

Embedded single-file reports can simply be opened in any browser. See Reports for what the report contains.

Cloud upload​

When TESTRELIC_API_KEY is set, runs upload to the TestRelic cloud at https://platform.testrelic.ai/api/v1. Cloud upload includes:

  • BrowserStack / LambdaTest detection — runs on those providers are detected and labeled automatically.
  • CI provider detection — GitHub Actions, GitLab CI, Jenkins, CircleCI, and Bitbucket Pipelines are detected automatically, with branch, commit, and run URL attached to the run.
  • Upload strategies — realtime or batch, selectable per run.
  • Offline queue — if the cloud is unreachable, uploads are written to .testrelic/queue/ and can be replayed later with testrelic-playwright drain.

Next steps​