Skip to main content
Ask AI

Cloud Quickstart

This walks you from a fresh install to an eval run visible in the TestRelic cloud.

1. Authenticate​

Either log in interactively:

Terminal
testrelic login

This saves your credentials to ~/.testrelic/credentials.toml. You can also pass values directly:

Terminal
testrelic login --api-key <key> --base-url <url>

Or skip the CLI entirely and set an environment variable:

Terminal
export TESTRELIC_API_KEY=<key>

Create a key from your TestRelic org — see API keys.

2. Run an evaluation​

Run your DeepEval suite as usual. The pytest plugin captures the run at session finish and uploads it:

Terminal
deepeval test run tests/

No flags or code changes are required. If credentials are missing, the upload simply no-ops — it never fails your tests.

3. See it in the cloud​

Once uploaded, the eval run surfaces in three places:

  • Eval workspace — the DeepEval / eval workspace, where it contributes to the Eval Stability metric.
  • Test Runs feed — eval runs appear in the unified Test Runs feed alongside your other runs.
  • Repo Evaluations tab — under the matching repository, in its Evaluations tab.

Open the latest run straight from your terminal:

Terminal
testrelic view

To open a specific run, pass its ID:

Terminal
testrelic view <run_id>
info

The cloud endpoint defaults to the TestRelic production endpoint, and your tr_* key authenticates against it. To target a different endpoint, set TESTRELIC_BASE_URL or TESTRELIC_CLOUD_ENDPOINT, or pass base_url= to the wrapper — the /evals path is resolved automatically. See Configuration.

Next steps​