First Test

This guide takes you from zero to a passing test. By the end you’ll have the CLI installed, a test written in YAML, and a report you can share. Time: ~10 minutes

Step 1: Install the CLI

curl -fsSL https://revyl.com/install.sh | sh

brew install RevylAI/tap/revyl

pipx install revyl

uv tool install revyl

pip install revyl

Verify it worked:

revyl version

Step 2: Authenticate

revyl auth login

You’ll be prompted for an API key. Get one from Account → API Keys.

Enter your API key: rvl_xxxxxxxxxxxxxxxxxxxx
✓ Authenticated as [email protected]
✓ Organization: My Company
✓ Credentials saved to ~/.revyl/credentials.json

For automated environments, set REVYL_API_KEY as an environment variable instead. See CI/CD Integration.

Step 3: Initialize your project

cd your-app
revyl init

The interactive wizard:

Detects your build system (Expo, Gradle, Xcode, Flutter, React Native)
Creates .revyl/config.yaml
Walks you through creating apps and uploading a build

To skip the wizard and configure manually:

revyl init -y

Step 4: Upload a build

If revyl init didn’t upload a build for you, do it now:

revyl build --platform android

Default debug builds work everywhere. If you’re uploading a custom build (release APK, .ipa, narrowed abiFilters), see Build Artifact Requirements.

Step 5: Write a YAML test

Create a file called login-smoke.yaml:

test:
  metadata:
    name: login-smoke
    platform: ios
    tags:
      - smoke
  build:
    name: my-ios-app
  blocks:
    - type: instructions
      step_description: Tap the Sign In button.
    - type: instructions
      step_description: Type "[email protected]" in the email field.
    - type: instructions
      step_description: Type "password123" in the password field.
    - type: instructions
      step_description: Tap Continue.
    - type: validation
      step_description: The home screen is visible.

Key fields:

test.metadata.name — must be unique in your org
test.metadata.platform — ios or android
test.build.name — must match a Revyl app name exactly. Check with revyl app list.
test.blocks — ordered list of steps (intent-level instructions, assertions in separate validation blocks)

Step 6: Create the test

revyl test create login-smoke --from-file ./login-smoke.yaml

This checks the YAML with backend validation, copies it to .revyl/tests/login-smoke.yaml, creates the remote test, and writes config if it doesn’t exist yet.

Step 8: Run the test

revyl test run login-smoke --open

The CLI queues the test, streams progress to your terminal, and opens the report in your browser.

Useful run flags

Flag	Description
`--open`	Open report in browser when complete
`--retries <n>`	Retry attempts (default: 1)
`--timeout <seconds>`	Maximum execution time (default: 3600)
`--no-wait`	Queue and exit immediately
`--json`	Structured JSON output
`--build-version-id <id>`	Pin a specific build version

Step 9: Iterate

Edit .revyl/tests/login-smoke.yaml, then push and re-run:

revyl test push login-smoke --force
revyl test run login-smoke

YAML as Source of Truth

For teams that want test definitions version-controlled alongside code.

Commit `.revyl/tests/` to git

The .revyl/tests/ directory is not gitignored by default. These YAML files are your source of truth — commit them.

git add .revyl/tests/
git commit -m "Add login-smoke test"

Daily sync pattern

# Start of day — pull any changes made in the browser editor
revyl test pull

# Work on tests locally in your IDE
# ...

# See what changed vs remote
revyl test diff login-smoke

# Push your changes
revyl test push

Check sync status

revyl test list

NAME              STATUS      PLATFORM   LAST MODIFIED
login-smoke       synced      ios        2 hours ago
checkout          modified    ios        5 minutes ago
onboarding        outdated    android    1 day ago

Status	Meaning
`synced`	Local and remote are identical
`modified`	Local changes not yet pushed
`outdated`	Remote has newer changes
`local-only`	Exists locally but not on remote

Reconcile when things drift

revyl sync --dry-run              # Preview what sync will change
revyl sync --tests --prune        # Reconcile and clean up stale mappings

Resolve conflicts

revyl test diff checkout          # See the diff
revyl test pull checkout --force  # Keep remote version
revyl test push checkout --force  # Keep local version

Quick Reference

Task	Command
See available tests	`revyl test list`
View a test report	`revyl test report login-smoke`
Open in browser editor	`revyl test open login-smoke`
Cancel a running test	`revyl test cancel <task_id>`
Check sync status	`revyl test list` (shows synced/modified/outdated)

Getting Started

Guides

CLI

Builds

Tests

Develop

Atlas

Integrations

Appendix

Step 1: Install the CLI

Step 2: Authenticate

Step 3: Initialize your project

Step 4: Upload a build

Step 5: Write a YAML test

Step 6: Create the test

Step 8: Run the test

Useful run flags

Step 9: Iterate

YAML as Source of Truth

Commit `.revyl/tests/` to git

Daily sync pattern

Check sync status

Reconcile when things drift

Resolve conflicts

Quick Reference

​Step 1: Install the CLI

​Step 2: Authenticate

​Step 3: Initialize your project

​Step 4: Upload a build

​Step 5: Write a YAML test

​Step 6: Create the test

​Step 8: Run the test

​Useful run flags

​Step 9: Iterate

​YAML as Source of Truth

​Commit .revyl/tests/ to git

​Daily sync pattern

​Check sync status

​Reconcile when things drift

​Resolve conflicts

​Quick Reference

Step 1: Install the CLI

Step 2: Authenticate

Step 3: Initialize your project

Step 4: Upload a build

Step 5: Write a YAML test

Step 6: Create the test

Step 8: Run the test

Useful run flags

Step 9: Iterate

YAML as Source of Truth

Commit `.revyl/tests/` to git

Daily sync pattern

Check sync status

Reconcile when things drift

Resolve conflicts

Quick Reference