> ## Documentation Index
> Fetch the complete documentation index at: https://docs.revyl.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run tests in CI

> Run Revyl tests on every pull request with GitHub Actions, GitLab CI, Buildkite, or the CLI in any pipeline.

Run Revyl tests automatically on every pull request. This guide covers GitHub Actions, the Revyl CLI in CI, and integrating test results into your workflow.

Prompt for your coding agent:

```text
Run this project's Revyl tests in CI on every pull request.

Read https://docs.revyl.com/tests/ci.md and follow it. It's the current Revyl guide for this, so prefer it over anything you remember about Revyl.

On GitHub, use GitHub Actions: put the critical tests in one `smoke-tests` workflow and run `revyl workflow run smoke-tests` on each pull request. Store `REVYL_API_KEY` as a CI secret; I'll add it myself.

You're done when a pull request runs the Revyl tests and shows their result.

Ask me before anything that needs my login or approval.
```

## Prerequisites

- Revyl account with an API key
- App and tests already created (see [Your First Test](/tests/first-test))

## Step 1: Store your API key

Add `REVYL_API_KEY` as a repository secret in GitHub:

**Settings → Secrets and variables → Actions → New repository secret**

Name: `REVYL_API_KEY`
Value: your API key from [Account → API Keys](https://auth.revyl.ai/account/api_keys)

## Step 2: Add a workflow file

### Option A: Run tests with saved choices

Run your workflow on every pull request with its saved choices. Tests saved on
Default branch run on the pull request's source branch when it has a usable
build, otherwise on the app's default branch. Tests created in the dashboard
start on All branches · Latest build instead; see
[New saved choices](/builds/branches#new-saved-choices), or use Option C.

```yaml
name: Revyl Tests

on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Revyl CLI
        run: |
          curl -fsSL https://revyl.com/install.sh | sh
          echo "$HOME/.revyl/bin" >> "$GITHUB_PATH"

      - name: Run smoke tests
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
        run: revyl workflow run smoke-tests
```

The installer downloads the native CLI binary. The CLI exits with code `0` on pass, `1` on failure.

### Option B: Build, upload, and test

Build your app fresh, upload the artifact, and run the full suite against it:

```yaml
name: Build and Test

on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Revyl CLI
        run: |
          curl -fsSL https://revyl.com/install.sh | sh
          echo "$HOME/.revyl/bin" >> "$GITHUB_PATH"

      - name: Build, upload, and run workflow
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
        run: revyl workflow run smoke-tests --build --platform android
```

`revyl workflow run <workflow> --build` builds your app on the runner with the
build settings in `.revyl/config.yaml`, uploads the artifact to the pull
request's source branch, and runs the workflow on that exact build.
`--platform` chooses the platform to build; tests for other platforms keep their
saved choices. Without `--build`, the workflow runs its saved choices, as in
Option A.

### Option C: Test an artifact from your own build job

After your CI build succeeds, upload its artifact and pass the returned build ID
to the workflow, so the run tests that exact build. The upload goes to the pull
request's source branch; see
[Choose where CI uploads go](/builds/ci-builds#choose-where-ci-uploads-go).

```yaml
- name: Install Revyl CLI
  run: |
    curl -fsSL https://revyl.com/install.sh | sh
    echo "$HOME/.revyl/bin" >> "$GITHUB_PATH"

- name: Upload Android build
  id: upload
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
    REVYL_ANDROID_APP_ID: ${{ vars.REVYL_ANDROID_APP_ID }}
  run: |
    revyl build upload --file ./build/app.apk \
      --app "$REVYL_ANDROID_APP_ID" --platform android \
      --yes --json > build.json
    BUILD_ID=$(jq -er 'select(.success == true and .count == 1) | .build.build_id | select(type == "string" and length > 0)' build.json)
    echo "build_id=$BUILD_ID" >> "$GITHUB_OUTPUT"

- name: Run workflow on the uploaded build
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
    WORKFLOW_ID: ${{ vars.WORKFLOW_ID }}
    REVYL_ANDROID_APP_ID: ${{ vars.REVYL_ANDROID_APP_ID }}
    REVYL_BUILD_ID: ${{ steps.upload.outputs.build_id }}
  run: revyl workflow run "$WORKFLOW_ID" --android-app "$REVYL_ANDROID_APP_ID" --android-build-id "$REVYL_BUILD_ID"
```

For one test, use `revyl test run "$TEST_ID" --build-id "$REVYL_BUILD_ID"`.
Use an existing test for this app/platform. Runtime app overrides replace the
workflow’s saved app configuration; unspecified platforms use each test’s saved
choice. An exact Android override does not select an iOS build.

### Option D: Use the Revyl GitHub Actions

Upload with `upload-build`, then run that build with `run-test`:

```yaml
- uses: RevylAI/revyl-gh-action/upload-build@main
  id: upload
  with:
    app-id: ${{ vars.REVYL_APP_ID }}
    version: ${{ github.event.pull_request.head.sha || github.sha }}
    file-path: ./app.apk
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
- uses: RevylAI/revyl-gh-action/run-test@main
  with:
    test-id: ${{ vars.REVYL_TEST_ID }}
    build-id: ${{ steps.upload.outputs.build-id }}
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
```

`upload-build` uploads to the pull request's source branch unless you set
`branch`. `run-test` and `run-workflow` follow the same branch rules as the CLI.
See the [Revyl GitHub Actions README](https://github.com/RevylAI/revyl-gh-action)
for every input.

## Step 3: Sync tests from CI

Keep remote tests in sync with your repo after merge to main:

```yaml
- name: Sync tests
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
  run: revyl test push --force
```

## CLI in CI (without GitHub Action)

Install and run the CLI directly in any CI:

```bash
curl -fsSL https://revyl.com/install.sh | sh
export PATH="$HOME/.revyl/bin:$PATH"
export REVYL_API_KEY=${{ secrets.REVYL_API_KEY }}

revyl test run login-flow --no-wait --json    # Queue and exit immediately
revyl workflow run smoke-tests --no-wait      # Queue a workflow
```

### Build upload from CI

```bash
revyl build upload --file build/App.app.zip --platform ios --yes    # Upload pre-built artifact
revyl build --profile release --platform android --json            # Build one recipe in the cloud
```

See [CI Builds](/builds/ci-builds#pattern-5-revyl-build-config-driven) for setup details.

## Useful CI Patterns

### Retries

```bash
revyl workflow run smoke-tests --retries 2
```

### Async execution

```bash
revyl workflow run regression --no-wait
```

### JSON output for downstream processing

```yaml
- name: Run with JSON output
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
  run: |
    result=$(revyl test run login-flow --json)
    echo "Report: $(echo $result | jq -r '.report_link')"
```

### Version-tagged builds

```yaml
- name: Build tagged with the commit
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
    COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
  run: revyl build --profile release --platform android --version "$COMMIT_SHA"
```

Pass both selectors in CI. No profile is stored as active/default, and a
non-interactive build fails with the available choices if selection remains
ambiguous.

### Keep main or staging current

Pull request builds don't refresh `main` or `staging` after merge. To keep a
branch as your source of truth, upload each new build to it and run your tests
on that build, explore it, or both, after every merge or on a schedule. See
[Keep a source-of-truth branch current](/builds/branches#keep-a-source-of-truth-branch-current).

## GitLab CI

```yaml
test:
  script:
    - curl -fsSL https://revyl.com/install.sh | sh
    - export PATH="$HOME/.revyl/bin:$PATH"
    - revyl workflow run smoke-tests
  variables:
    REVYL_API_KEY: $REVYL_API_KEY
```

To test a merge request's own build on GitLab, see
[Choose where CI uploads go](/builds/ci-builds#choose-where-ci-uploads-go).

## Buildkite

```yaml
steps:
  - label: "Revyl tests"
    command: |
      curl -fsSL https://revyl.com/install.sh | sh
      export PATH="$$HOME/.revyl/bin:$$PATH"
      revyl build upload --file ./app.apk --app "$$REVYL_ANDROID_APP_ID" \
        --platform android --yes --json > build.json
      BUILD_ID=$$(jq -er '.build.build_id' build.json)
      revyl workflow run smoke-tests --android-app "$$REVYL_ANDROID_APP_ID" \
        --android-build-id "$$BUILD_ID"
```

Provide `REVYL_API_KEY` and `REVYL_ANDROID_APP_ID` as Buildkite secrets or
environment variables. `$$` defers each variable to run time.

## CI-Friendly Flags

See [CI-Friendly Flags](/builds/ci-builds#ci-friendly-flags).

## Exit Codes

| Code | Meaning |
|------|---------|
| `0` | Test/workflow passed |
| `1` | Test/workflow failed or error |

## Environment Variables

| Variable | Description |
|----------|-------------|
| `REVYL_API_KEY` | API key for authentication (required in CI) |