GitHub Actions - Revyl Documentation

Overview

The official Revyl GitHub Action makes it easy to integrate automated testing into your GitHub workflows. Run tests on every pull request, commit, or schedule to catch bugs early and maintain code quality. Repository: https://github.com/RevylAI/revyl-gh-action

Features

✅ Build Upload - Upload builds directly from CI with automatic metadata
✅ Build-to-Test Pipelines - Test the exact build you just created
✅ Test & Workflow Support - Run individual tests or complete workflows
✅ Real-time Monitoring - Watch tests execute with SSE streaming
✅ Rich Outputs - Get task IDs, report links, and execution details
✅ Automatic Retries - Configure retry behavior for flaky tests
✅ Framework Agnostic - Works with React Native, Expo, Flutter, native builds

Quick Start

1. Get Your API Key

First, create a Revyl API key:

Go to Account → Personal API Keys in Revyl
Click New API key
Select an expiration period (recommended: 1 year for CI/CD)
Click Create and copy the key

See the CI/CD Overview for detailed instructions with screenshots.

2. Add API Key to GitHub Secrets

Go to your GitHub repository
Navigate to Settings → Secrets and variables → Actions
Click New repository secret
Name: REVYL_API_KEY
Value: Paste your API key
Click Add secret

Never commit API keys directly in your workflow files! Always use GitHub Secrets.

3. Get Your Workflow ID

Open your workflow in Revyl
Click the Copy workflow ID button next to the workflow name
Or copy the ID from the URL: https://app.revyl.ai/workflow/{workflow-id}

Workflow page showing workflow ID and copy button

Workflows are the recommended way to run tests in CI/CD. They provide comprehensive monitoring, support multiple tests, and give better execution insights than running individual tests.

Don’t want to set up GitHub Actions? Enable Manual Trigger in your GitHub repo configuration to use @revyl commands directly in PR comments. Just type @revyl config run to trigger tests.

4. Create Your Workflow File

Create .github/workflows/revyl-tests.yml in your repository:

name: Revyl Tests

on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Run Revyl Workflow
        uses: RevylAI/revyl-gh-action/run-workflow@main
        with:
          workflow-id: "your-workflow-id-here"
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}

Build-to-Test Pipeline

The most powerful way to use Revyl Actions - automatically test your freshly built apps:

name: Build and Test Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      # Your build steps here (React Native, Expo, Flutter, etc.)

      - name: Upload Build to Revyl
        id: upload-build
        uses: RevylAI/revyl-gh-action/upload-build@main
        with:
          build-var-id: ${{ vars.BUILD_VAR_ID }}
          version: ${{ github.sha }}
          file-path: path/to/your/app.apk
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}

      - name: Run Workflow on New Build
        uses: RevylAI/revyl-gh-action/run-workflow@main
        with:
          workflow-id: ${{ vars.WORKFLOW_ID }}
          build-version-id: ${{ steps.upload-build.outputs.version-id }}
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}

Expo Build Pipeline

For Expo/EAS builds, use the expo-url input (.tar.gz files are automatically converted):

- name: Upload Expo Build
  uses: RevylAI/revyl-gh-action/upload-build@main
  with:
    build-var-id: ${{ vars.BUILD_VAR_ID }}
    version: ${{ github.sha }}
    expo-url: "https://expo.dev/artifacts/eas/..."
    expo-headers: '{"Authorization": "Bearer ${{ secrets.EXPO_TOKEN }}"}'
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}

Automatic CI/CD Metadata

The upload-build action automatically injects CI/CD metadata into every build - no configuration required:

ci_run_url: Direct link to the GitHub Actions run
commit_sha: Git commit SHA that triggered the build
branch: Branch name where the build was triggered
pr_number: Pull request number (for PR builds)
ci_system: github-actions
ci_build_number: GitHub run number

This provides excellent traceability without any manual setup.

Configuration Options

Environment Variables

Variable	Required	Description
`REVYL_API_KEY`	Yes	API key for authentication. Get from Personal API Keys

Run Test/Workflow Inputs

Input	Required	Default	Description
`test-id`	Yes (if no `workflow-id`)	N/A	The ID of the test to run
`workflow-id`	Yes (if no `test-id`)	N/A	The ID of the workflow to run
`build-version-id`	No	N/A	Specific build version to test (from upload)
`timeout`	No	`3600`	Timeout in seconds (default: 1 hour)
`retries`	No	`1`	Number of retry attempts if the test fails

We recommend using run-workflow with workflow-id for CI/CD. Workflows provide better monitoring, support multiple tests, and give richer execution insights. Use run-test with test-id only for simple single-test scenarios.

Upload Build Inputs

Input	Required	Default	Description
`build-var-id`	Yes	N/A	Build variable ID from Revyl
`version`	Yes	N/A	Version string (e.g., `1.0.0`, commit SHA)
`file-path`	Yes (if no `expo-url`)	N/A	Path to local build file (.apk, .app)
`expo-url`	Yes (if no `file-path`)	N/A	Expo/EAS build artifact URL
`expo-headers`	No	N/A	JSON headers for Expo URL (auth tokens)

Action Outputs

Both actions provide outputs for integration with other workflow steps.

Upload Build Outputs

Output	Description
`success`	Whether upload was successful
`version-id`	ID of created build version (use for `build-version-id`)
`version`	Version string of uploaded build
`package-id`	Extracted package ID from build
`upload-time`	Time taken for upload in seconds
`error-message`	Error message if upload failed

Run Test Outputs

Output	Description
`success`	Whether test completed successfully
`task_id`	Unique task ID for the execution
`execution_time`	Total execution time in seconds
`platform`	Platform the test ran on
`report_link`	Shareable link to test report
`total_steps`	Total number of test steps
`completed_steps`	Number of completed steps
`error_message`	Error message if execution failed

Workflow-specific Outputs

Output	Description
`total_tests`	Total number of tests in workflow
`completed_tests`	Number of tests completed
`passed_tests`	Number of tests that passed
`failed_tests`	Number of tests that failed

Using Outputs in Subsequent Steps

- name: Upload Build
  id: upload
  uses: RevylAI/revyl-gh-action/upload-build@main
  with:
    build-var-id: ${{ vars.BUILD_VAR_ID }}
    version: ${{ github.sha }}
    file-path: ./app.apk
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}

- name: Run Workflow with New Build
  id: workflow
  uses: RevylAI/revyl-gh-action/run-workflow@main
  with:
    workflow-id: ${{ vars.WORKFLOW_ID }}
    build-version-id: ${{ steps.upload.outputs.version-id }}
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}

- name: Share Results
  if: always()
  run: |
    echo "Success: ${{ steps.workflow.outputs.success }}"
    echo "Total Tests: ${{ steps.workflow.outputs.total_tests }}"
    echo "Passed: ${{ steps.workflow.outputs.passed_tests }}"

Common Workflows

Pull Request Testing

Run workflows automatically when PRs are opened or updated:

name: PR Tests

on:
  pull_request:
    branches: [main, develop]
    types: [opened, reopened, synchronize]

permissions:
  contents: read

jobs:
  revyl-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Run Revyl Workflow
        uses: RevylAI/revyl-gh-action/run-workflow@main
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
        with:
          workflow-id: "your-workflow-id"
          retries: 2

Staging Branch Validation

Only run workflows when PRs are from the staging branch:

name: Staging Validation

on:
  pull_request:
    branches: [main]
    types: [opened, reopened, synchronize]

permissions:
  contents: read

jobs:
  revyl-smoke-tests:
    # Only run if PR is from staging branch
    if: github.event.pull_request.head.ref == 'staging'
    runs-on: ubuntu-latest
    steps:
      - name: Run Smoke Tests
        uses: RevylAI/revyl-gh-action/run-workflow@main
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
        with:
          workflow-id: "your-workflow-id"

Scheduled Regression Testing

Run comprehensive test suites on a schedule:

name: Nightly Regression

on:
  schedule:
    # Run every night at 2 AM UTC
    - cron: "0 2 * * *"
  # Allow manual trigger
  workflow_dispatch:

jobs:
  regression-suite:
    runs-on: ubuntu-latest
    steps:
      - name: Run Full Regression Suite
        uses: RevylAI/revyl-gh-action/run-workflow@main
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
        with:
          workflow-id: "regression-workflow-id"
          retries: 3

Pre-Deployment Gate

Prevent deployments if workflows fail:

name: Deploy with Testing

on:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Run Pre-Deployment Workflow
        uses: RevylAI/revyl-gh-action/run-workflow@main
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
        with:
          workflow-id: "smoke-workflow-id"
          retries: 2

  deploy:
    needs: test # Only deploy if workflow passes
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Deploy to Production
        run: |
          echo "Deploying to production..."
          ./deploy.sh

Advanced Configuration

Retry Configuration

Configure retry behavior for flaky tests:

- name: Run Workflow with Retries
  uses: RevylAI/revyl-gh-action/run-workflow@main
  env:
    REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
  with:
    workflow-id: "your-workflow-id"
    retries: 5 # Retry up to 5 times

Troubleshooting

Test Fails Immediately

Problem: Test fails with authentication error Solution:

Verify your API key is correct in GitHub Secrets
Check the key hasn’t expired
Ensure the key has permissions to run tests

Test Times Out

Problem: Test runs but never completes Solution:

Check the device is available in Revyl dashboard
Increase the timeout in your workflow (default is usually sufficient)
Verify the test runs successfully when triggered manually in Revyl

Flaky Tests

Problem: Tests pass/fail inconsistently Solution:

Increase the retries parameter
Review test stability in Revyl
Add explicit waits in your test steps
Check for race conditions or timing issues

Custom Device URL Not Working

Problem: Test doesn’t use the specified device URL Solution:

Verify the URL is correct and accessible
Check that the device is online and available
Ensure the device supports your test’s platform (Android or iOS)

Best Practices

1. Use Organization Secrets

For team projects, use organization-level secrets instead of repository secrets:

Go to your GitHub Organization settings
Navigate to Secrets and variables → Actions
Add REVYL_API_KEY as an organization secret
Select which repositories can access it

2. Separate Test Environments

Use different workflows for different environments:

jobs:
  test-dev:
    if: github.base_ref == 'develop'
    steps:
      - uses: RevylAI/revyl-gh-action/run-workflow@main
        with:
          workflow-id: "dev-workflow-id"
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}

  test-prod:
    if: github.base_ref == 'main'
    steps:
      - uses: RevylAI/revyl-gh-action/run-workflow@main
        with:
          workflow-id: "prod-workflow-id"
        env:
          REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}

3. Conditional Execution

Save CI time by only running tests when relevant files change:

on:
  pull_request:
    paths:
      - "src/**"
      - "tests/**"
      - ".github/workflows/revyl-tests.yml"

Getting Started

Tests

Builds

Workflows

Monitoring

Integrations

CLI

CI/CD

Device Automation

​Overview

​Features

​Quick Start

​1. Get Your API Key

​2. Add API Key to GitHub Secrets

​3. Get Your Workflow ID

​4. Create Your Workflow File

​Build-to-Test Pipeline

​Expo Build Pipeline

​Automatic CI/CD Metadata

​Configuration Options

​Environment Variables

​Run Test/Workflow Inputs

​Upload Build Inputs

​Action Outputs

​Upload Build Outputs

​Run Test Outputs

​Workflow-specific Outputs

​Using Outputs in Subsequent Steps

​Common Workflows

​Pull Request Testing

​Staging Branch Validation

​Scheduled Regression Testing

​Pre-Deployment Gate

​Advanced Configuration

​Retry Configuration

​Troubleshooting

​Test Fails Immediately

​Test Times Out

​Flaky Tests

​Custom Device URL Not Working

​Best Practices

​1. Use Organization Secrets

​2. Separate Test Environments

​3. Conditional Execution

​Next Steps

​Resources

Overview

Features

Quick Start

1. Get Your API Key

2. Add API Key to GitHub Secrets

3. Get Your Workflow ID

4. Create Your Workflow File

Build-to-Test Pipeline

Expo Build Pipeline

Automatic CI/CD Metadata

Configuration Options

Environment Variables

Run Test/Workflow Inputs

Upload Build Inputs

Action Outputs

Upload Build Outputs

Run Test Outputs

Workflow-specific Outputs

Using Outputs in Subsequent Steps

Common Workflows

Pull Request Testing

Staging Branch Validation

Scheduled Regression Testing

Pre-Deployment Gate

Advanced Configuration

Retry Configuration

Troubleshooting

Test Fails Immediately

Test Times Out

Flaky Tests

Custom Device URL Not Working

Best Practices

1. Use Organization Secrets

2. Separate Test Environments

3. Conditional Execution

Next Steps

Resources