Skip to main content

​
Your First Test

This guide walks you through creating your first Revyl test from scratch, then shows how to set up YAML-based test management for your team.

​
Part A: Create and Run a Test

​
1. Install the CLI

brew install RevylAI/tap/revyl    # Homebrew (macOS)
pipx install revyl                # pipx (cross-platform)
uv tool install revyl             # uv
pip install revyl                 # pip

​
2. Authenticate

revyl auth login
You’ll be prompted for an API key. Get one from Account → API Keys.

​
3. Initialize your project

cd your-app
revyl init
The interactive wizard:
  1. Detects your build system (Expo, Gradle, Xcode, Flutter, React Native)
  2. Creates .revyl/config.yaml
  3. Walks you through creating apps, uploading a build, and creating your first test
If you prefer to skip the wizard and configure manually: 
revyl init -y

​
4. Write a YAML test file

Create a file called login-smoke.yaml: 
test:
  metadata:
    name: login-smoke
    platform: ios
    tags:
      - smoke
  build:
    name: my-ios-app          # Must match a Revyl app name (check with: revyl app list)
  blocks:
    - type: instructions
      step_description: Tap the Sign In button.
    - type: instructions
      step_description: Type "[email protected]" in the email field.
    - type: instructions
      step_description: Type "password123" in the password field.
    - type: instructions
      step_description: Tap Continue.
    - type: validation
      step_description: The home screen is visible.
Key fields:
  • test.metadata.name — the test name (must be unique in your org)
  • test.metadata.platform — ios or android
  • test.build.name — must match a Revyl app name exactly. Check with revyl app list.
  • test.blocks — ordered list of steps

​
5. Validate the YAML

revyl test validate ./login-smoke.yaml
Fix any errors before proceeding. Use --json for machine-readable output in CI.

​
6. Create the test

revyl test create login-smoke --from-file ./login-smoke.yaml
This:
  • Validates the YAML
  • Copies it to .revyl/tests/login-smoke.yaml
  • Creates (or updates) the remote test
  • Writes .revyl/config.yaml if it doesn’t exist yet

​
7. Run the test

revyl test run login-smoke --open
The CLI queues the test, streams progress, and opens the report in your browser when done.

​
8. Iterate

Edit .revyl/tests/login-smoke.yaml, then push and re-run: 
revyl test push login-smoke --force
revyl test run login-smoke

​
Part B: YAML as Source of Truth

For teams that want test definitions version-controlled alongside code.

​
Commit .revyl/tests/ to git

The .revyl/tests/ directory is not gitignored by default. These YAML files are your source of truth — commit them. 
git add .revyl/tests/
git commit -m "Add login-smoke test"

​
Daily sync pattern

# Start of day — pull any changes made in the browser editor
revyl test pull

# Work on tests locally in your IDE
# ...

# See what changed vs remote
revyl test diff login-smoke

# Push your changes
revyl test push

​
Check sync status

revyl test list

NAME              STATUS      PLATFORM   LAST MODIFIED
login-smoke       synced      ios        2 hours ago
checkout          modified    ios        5 minutes ago
onboarding        outdated    android    1 day ago
StatusMeaning
syncedLocal and remote are identical
modifiedLocal changes not yet pushed
outdatedRemote has newer changes
local-onlyExists locally but not on remote

​
Reconcile when things drift

revyl sync --dry-run              # Preview what sync will change
revyl sync --tests --prune        # Reconcile and clean up stale mappings

​
PR-based test changes

Edit YAML in a feature branch, review the diff in a pull request like any code change. After merge, push to remote: 
revyl test push --force
Or automate in CI: 
revyl test push --force && revyl test run login-smoke

​
Resolve conflicts

# See the diff
revyl test diff checkout

# Keep remote version
revyl test pull checkout --force

# Keep local version
revyl test push checkout --force

​
What’s Next