Run Tests within CI/CD Workflow - Revyl Documentation

Run Revyl tests automatically on every pull request. This guide covers GitHub Actions, the Revyl CLI in CI, and integrating test results into your workflow.

Prerequisites

Revyl account with an API key
App and tests already created (see Your First Test)

Step 1: Store your API key

Add REVYL_API_KEY as a repository secret in GitHub: Settings → Secrets and variables → Actions → New repository secret Name: REVYL_API_KEY Value: your API key from Account → API Keys

Step 2: Add a workflow file

Option A: Run tests against the latest build

The simplest setup. Runs your test workflow on every PR using the most recently uploaded build.

name: Revyl Tests

on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Revyl CLI
        run: |
          curl -fsSL https://revyl.com/install.sh | sh
          echo "$HOME/.revyl/bin" >> "$GITHUB_PATH"

      - name: Run smoke tests
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
        run: revyl workflow run smoke-tests

The installer downloads the native CLI binary. The CLI exits with code 0 on pass, 1 on failure.

Option B: Build, upload, and test

Build your app fresh, upload the artifact, and run the full suite against it:

name: Build and Test

on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Revyl CLI
        run: |
          curl -fsSL https://revyl.com/install.sh | sh
          echo "$HOME/.revyl/bin" >> "$GITHUB_PATH"

      - name: Build, upload, and run workflow
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
        run: revyl run smoke-tests -w --platform android

revyl run <workflow> -w builds your app, uploads the artifact, and runs all tests. Use --no-build to skip the build step.

Option C: Use the Revyl CLI in GitHub Actions

- name: Install Revyl CLI
  run: curl -fsSL https://revyl.com/install.sh | sh

- name: Upload Android build to Revyl
  run: |
    revyl build upload \
      --file ./build/app.apk \
      --app "$REVYL_ANDROID_APP_ID" \
      --platform android \
      --version "$REVYL_PR_HEAD_SHA" \
      --yes \
      --json
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
    REVYL_ANDROID_APP_ID: ${{ vars.REVYL_ANDROID_APP_ID }}
    REVYL_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}

Build-to-test pipeline (GitHub Actions)

Upload a build and test it in the same workflow:

- name: Upload Build
  id: upload
  run: |
    revyl build upload \
      --file ./build/app.apk \
      --app "$REVYL_ANDROID_APP_ID" \
      --platform android \
      --version "$REVYL_PR_HEAD_SHA" \
      --yes \
      --json
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
    REVYL_ANDROID_APP_ID: ${{ vars.REVYL_ANDROID_APP_ID }}
    REVYL_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}

- name: Run Workflow on Current Build
  run: revyl workflow run "$WORKFLOW_ID" --android-app "$REVYL_ANDROID_APP_ID"
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
    WORKFLOW_ID: ${{ vars.WORKFLOW_ID }}
    REVYL_ANDROID_APP_ID: ${{ vars.REVYL_ANDROID_APP_ID }}

Generate an Atlas map in CI

Use Explore when a workflow needs to confirm that an uploaded app can run and produce a usable Atlas map:

- name: Explore app and generate Atlas map
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
    REVYL_ANDROID_APP_ID: ${{ vars.REVYL_ANDROID_APP_ID }}
  run: |
    revyl explore run "$REVYL_ANDROID_APP_ID" \
      --explorers 3 \
      --strategy surface-sweep \
      --max-duration 20m \
      --timeout 30m \
      --json > explore-result.json

- name: Upload Explore result
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: revyl-explore-result
    path: explore-result.json

Store authentication values in Revyl and attach them with --launch-var; do not pass secrets through --launch-env. A usable partial map and setup blockers exit successfully. A failed, cancelled, timed-out, or no-map run exits non-zero. Explore gates app execution and map generation, not product correctness. Keep tests or workflows in the same pipeline when a change must satisfy behavioral assertions. See Explore from the CLI for the full outcome contract.

Step 3: Sync tests from CI

Keep remote tests in sync with your repo after merge to main:

- name: Sync tests
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
  run: revyl test push --force

CLI in CI (without GitHub Action)

Install and run the CLI directly in any CI:

curl -fsSL https://revyl.com/install.sh | sh
export PATH="$HOME/.revyl/bin:$PATH"
export REVYL_API_KEY=${{ secrets.REVYL_API_KEY }}

revyl test run login-flow --no-wait --json    # Queue and exit immediately
revyl workflow run smoke-tests --no-wait      # Queue a workflow

Build upload from CI

revyl build upload --file build/App.app.zip --platform ios --yes    # Upload pre-built artifact
revyl build --platform android --json                              # Build and upload from config

Useful CI Patterns

Retries

revyl workflow run smoke-tests --retries 2

Async execution

revyl workflow run regression --no-wait

JSON output for downstream processing

- name: Run with JSON output
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
  run: |
    result=$(revyl test run login-flow --json)
    echo "Report: $(echo $result | jq -r '.report_link')"

Version-tagged builds

revyl build --platform android --version "$REVYL_PR_HEAD_SHA"

GitLab CI

test:
  script:
    - curl -fsSL https://revyl.com/install.sh | sh
    - export PATH="$HOME/.revyl/bin:$PATH"
    - revyl workflow run smoke-tests
  variables:
    REVYL_API_KEY: $REVYL_API_KEY

CI-Friendly Flags

Flag	Effect
`--json`	Machine-readable JSON output
`--no-wait`	Queue the run and exit without waiting for results
`--quiet` / `-q`	Suppress non-essential output

Exit Codes

Code	Meaning
`0`	Test/workflow passed
`1`	Test/workflow failed or error

Environment Variables

Variable	Description
`REVYL_API_KEY`	API key for authentication (required in CI)

​Prerequisites

​Step 1: Store your API key

​Step 2: Add a workflow file

​Option A: Run tests against the latest build

​Option B: Build, upload, and test

​Option C: Use the Revyl CLI in GitHub Actions

​Build-to-test pipeline (GitHub Actions)

​Generate an Atlas map in CI

​Step 3: Sync tests from CI

​CLI in CI (without GitHub Action)

​Build upload from CI

​Useful CI Patterns

​Retries

​Async execution

​JSON output for downstream processing

​Version-tagged builds

​GitLab CI

​CI-Friendly Flags

​Exit Codes

​Environment Variables