Run Tests within CI/CD Workflow

Run Revyl tests automatically on every pull request. This guide covers GitHub Actions, the Revyl CLI in CI, and integrating test results into your workflow.

Prerequisites

Revyl account with an API key
App and tests already created (see Your First Test)

Step 1: Store your API key

Add REVYL_API_KEY as a repository secret in GitHub: Settings → Secrets and variables → Actions → New repository secret Name: REVYL_API_KEY Value: your API key from Account → API Keys

Step 2: Add a workflow file

Option A: Run tests against the latest build

The simplest setup. Runs your test workflow on every PR using the most recently uploaded build.

name: Revyl Tests

on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Revyl CLI
        run: |
          curl -fsSL https://revyl.com/install.sh | sh
          echo "$HOME/.revyl/bin" >> "$GITHUB_PATH"

      - name: Run smoke tests
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
        run: revyl workflow run smoke-tests

The installer downloads the native CLI binary. The CLI exits with code 0 on pass, 1 on failure.

Option B: Build, upload, and test

Build your app fresh, upload the artifact, and run the full suite against it:

name: Build and Test

on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Revyl CLI
        run: |
          curl -fsSL https://revyl.com/install.sh | sh
          echo "$HOME/.revyl/bin" >> "$GITHUB_PATH"

      - name: Build, upload, and run workflow
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
        run: revyl run smoke-tests -w --platform android

revyl run <workflow> -w builds your app, uploads the artifact, and runs all tests. Use --no-build to skip the build step.

Option C: Use the Revyl CLI in GitHub Actions

- name: Install Revyl CLI
  run: curl -fsSL https://revyl.com/install.sh | sh

- name: Upload Android build to Revyl
  run: |
    revyl build upload \
      --file ./build/app.apk \
      --app "$REVYL_ANDROID_APP_ID" \
      --platform android \
      --version "$REVYL_PR_HEAD_SHA" \
      --yes \
      --json
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
    REVYL_ANDROID_APP_ID: ${{ vars.REVYL_ANDROID_APP_ID }}
    REVYL_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}

Build-to-test pipeline (GitHub Actions)

Upload a build and test it in the same workflow:

- name: Upload Build
  id: upload
  run: |
    revyl build upload \
      --file ./build/app.apk \
      --app "$REVYL_ANDROID_APP_ID" \
      --platform android \
      --version "$REVYL_PR_HEAD_SHA" \
      --yes \
      --json
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
    REVYL_ANDROID_APP_ID: ${{ vars.REVYL_ANDROID_APP_ID }}
    REVYL_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}

- name: Run Workflow on Current Build
  run: revyl workflow run "$WORKFLOW_ID" --android-app "$REVYL_ANDROID_APP_ID"
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
    WORKFLOW_ID: ${{ vars.WORKFLOW_ID }}
    REVYL_ANDROID_APP_ID: ${{ vars.REVYL_ANDROID_APP_ID }}

Step 3: Sync tests from CI

Keep remote tests in sync with your repo after merge to main:

- name: Sync tests
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
  run: revyl test push --force

CLI in CI (without GitHub Action)

Install and run the CLI directly in any CI:

curl -fsSL https://revyl.com/install.sh | sh
export PATH="$HOME/.revyl/bin:$PATH"
export REVYL_API_KEY=${{ secrets.REVYL_API_KEY }}

revyl test run login-flow --no-wait --json    # Queue and exit immediately
revyl workflow run smoke-tests --no-wait      # Queue a workflow

Build upload from CI

revyl build upload --file build/App.app.zip --platform ios --yes    # Upload pre-built artifact
revyl build --platform android --json                              # Build and upload from config

Useful CI Patterns

Retries

revyl workflow run smoke-tests --retries 2

Async execution

revyl workflow run regression --no-wait

JSON output for downstream processing

- name: Run with JSON output
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
  run: |
    result=$(revyl test run login-flow --json)
    echo "Report: $(echo $result | jq -r '.report_link')"

Version-tagged builds

revyl build --platform android --version "$REVYL_PR_HEAD_SHA"

GitLab CI

test:
  script:
    - curl -fsSL https://revyl.com/install.sh | sh
    - export PATH="$HOME/.revyl/bin:$PATH"
    - revyl workflow run smoke-tests
  variables:
    REVYL_API_KEY: $REVYL_API_KEY

CI-Friendly Flags

Flag	Effect
`--json`	Machine-readable JSON output
`--no-wait`	Queue the run and exit without waiting for results
`--quiet` / `-q`	Suppress non-essential output

Exit Codes

Code	Meaning
`0`	Test/workflow passed
`1`	Test/workflow failed or error

Environment Variables

Variable	Description
`REVYL_API_KEY`	API key for authentication (required in CI)

Getting Started

Guides

CLI

Builds

Tests

Develop

Atlas

Integrations

Appendix

Prerequisites

Step 1: Store your API key

Step 2: Add a workflow file

Option A: Run tests against the latest build

Option B: Build, upload, and test

Option C: Use the Revyl CLI in GitHub Actions

Build-to-test pipeline (GitHub Actions)

Step 3: Sync tests from CI

CLI in CI (without GitHub Action)

Build upload from CI

Useful CI Patterns

Retries

Async execution

JSON output for downstream processing

Version-tagged builds

GitLab CI

CI-Friendly Flags

Exit Codes

Environment Variables

​Prerequisites

​Step 1: Store your API key

​Step 2: Add a workflow file

​Option A: Run tests against the latest build

​Option B: Build, upload, and test

​Option C: Use the Revyl CLI in GitHub Actions

​Build-to-test pipeline (GitHub Actions)

​Step 3: Sync tests from CI

​CLI in CI (without GitHub Action)

​Build upload from CI

​Useful CI Patterns

​Retries

​Async execution

​JSON output for downstream processing

​Version-tagged builds

​GitLab CI

​CI-Friendly Flags

​Exit Codes

​Environment Variables

Prerequisites

Step 1: Store your API key

Step 2: Add a workflow file

Option A: Run tests against the latest build

Option B: Build, upload, and test

Option C: Use the Revyl CLI in GitHub Actions

Build-to-test pipeline (GitHub Actions)

Step 3: Sync tests from CI

CLI in CI (without GitHub Action)

Build upload from CI

Useful CI Patterns

Retries

Async execution

JSON output for downstream processing

Version-tagged builds

GitLab CI

CI-Friendly Flags

Exit Codes

Environment Variables