Runs and results
A run is one flow on one device, once. Every run is recorded, so when one fails you can see exactly what happened, step by step.
What's recorded
- The recording, in step with the flow's steps: choose a step to jump to it.
- The steps, each with how long it took, and the one that failed with what went wrong.
- The screen at the failure, and what was on it, which Trailcam AI reads to explain why.
- The device's log, searchable, with the lines from the failure marked.
- The app's requests, under the steps: where each went, how it answered and how long it took, at the step it came in (below).
- A JUnit report, to download for your own tools.
How long recordings, screenshots, logs and requests are kept depends on the plan: 7 days on Free, 30 on Starter, 90 on Team and 180 on Business. The run's result is kept after its recording goes.
Results
| Result | What it means |
|---|---|
| Passed | Every step did what the flow says. |
| Failed | A step didn't: something wasn't on the screen, a check wasn't true, the app crashed, or the flow ran out of time. |
| Couldn't run | Trailcam couldn't run it, or the build can't run on our devices. It's not a result for your app, and it's free. |
| Stopped | Someone stopped it, or CI cancelled the job. |
While a run waits for a device it shows as waiting, then as running.
When a run fails
The run's page says what went wrong in plain words, under the step where it happened, with the device's own message and what to try. Common causes:
- It couldn't find something: the text or element wasn't on the screen before the step's time ran out. The screenshot shows what was there instead.
- A check failed:
assertVisible,assertTrueor an AI check wasn't true. - The app crashed: confirmed from the device's crash report, not guessed.
- It took too long: the flow passed 15 minutes.
When it's our problem
If a device fails, an install fails or a recording doesn't upload, Trailcam retries the run on a fresh device, up to three times, free. The run's page says "Retried on a fresh device". If it still can't run, it shows Couldn't run, which never counts toward your runs.
The app's requests
Was it the app or the server? Every run records the requests your app made, listed under the steps: when each started (on the recording's time), the step it came in, where it went, how it answered and how long it took. A request that failed or answered with an error is flagged, and Problems shows only those. A request that took a second or more says where its time went: finding the server, connecting, sending, waiting for the answer and receiving it. Choose a request to play the recording from when it started. Download has the whole list as a HAR file, which Chrome's developer tools, Charles and Proxyman open, and as JSON.
What's kept is where each request went and how it went, never what it carried: no headers, no bodies, and an address's query keeps its names but not their values. Nothing changes in your app or its traffic.
- On iOS, Trailcam notes each request your app makes through iOS's own networking (URLSession, which native apps, React Native and most SDKs use), in the simulator only, with iOS's own measurement of its phases. Nothing is added to your build.
- On Android, the emulator's connections go through Trailcam, which sees plain HTTP requests in full and each HTTPS connection by its server: when, how long, how much went each way, how long it took to connect, and how it ended (no answer, refused, reset). Android's own connections, like its connectivity checks, are hidden unless you choose Show Android's own.
When a run fails because a request did, Trailcam AI says so, and names the request.
Accessibility
Every run checks the screens its journey passes through for what keeps people out, with nothing to set up or write.
- On Android, by Google's Accessibility Test Framework, the checks Accessibility Scanner and Espresso run. It finds what has no words for TalkBack, what's smaller than 48 × 48 dp to tap, words with too little contrast, and more: a text field described instead of labelled, two things to tap in one place, something TalkBack can't say the kind of. It checks each screen as the run captured it, which doesn't include text sizes, so all words are held to 3 to 1, the bar for large words.
- On iOS, by Trailcam's own checks: what has no words for VoiceOver, what's smaller than 44 × 44 points to tap, and words with too little contrast against their background, under 4.5 to 1 (3 to 1 for large words), as WCAG asks. Apple's own accessibility audit can't run beside a journey: it works the app at the same time as the journey's steps, which slowed them and, in our tests, made flows fail.
The run's page lists each screen with a problem: a picture of it with the problem outlined, when it showed (choose it to play the recording from there), and what's wrong, in numbers. The flow's page keeps the journey's report on each platform: its latest run's problems, which are new since the latest run on an earlier build, and which that build had and this one fixed.
It never fails a run: the problems are for your team to fix, not a gate. What can't be measured fairly is left out rather than guessed at, whichever checks found it: anything cut off by the screen or a scrolling list, anything under the keyboard or a bar (a tab bar, a toolbar), and on iOS the height of a text field, which iOS reports as its line of words rather than the box that focuses it.
Long runs
A run counts once for every 3 minutes of its flow, started: up to 3 minutes is one run, and 12 minutes is four. Getting the device ready isn't counted. A run's page shows how long it took and how it counted.
Sharing a run
Share makes a link to the run for someone without a Trailcam account, like a developer on another team. Links last 1, 7 or 30 days, include the log (and with it, the app's requests) only if you choose, and can be turned off at any time. The shared page shows the recording, the steps and what went wrong, but not who started the run.