Test reports
Test reports sum up the results of a test run - a single execution of all your active test cases.
Where to view your test reports
You will find a summary of the most recent 4 test runs in the test reports section in the project overview. Click on show more to see more than the last 4 test runs. The test reports menu in the sidebar lists all recent reports in chronological order, starting from most recent at the top.
Test reports are tied to a single target execution URL, so one specific deployment of your website or app, e.g. a branch deployment during a pull request.
'test report' list in the 'project overview' in the app, 08/2024
Scheduled test runs
If you want to run your tests on a fixed schedule, the test reports section in the project overview gives you the option of a daily, weekly or bi-weekly interval.
This ensures that your app works flawlessly over time. We will send you an email notification if any of your tests fail.
scheduling test runs, 04/2024
Test reports in your CI/CD pipeline
If you integrated Octomind into your CI pipeline, we will comment test results back in your pull request after a completed test run.
Example of Octomind test results in a commit comment, 09/2023
There will be a row for each test with the following information:
| column | description |
|---|---|
| description | name of the test case |
| status ✅ | execution successful |
| status ⌛️ | execution in progress |
| status ❌ | execution failed |
| status 💔 | something went wrong on our side during execution |
| details | link which takes you to the test case details (see below) |
Test report details
When you click on a specific test report in the app or on the link in your commit comment, a test report overview with all the results and more details will open.
If trigger manually, you will find the user who triggered it and when. When triggered via CI integration, you will find a link to the commit that triggered it.
Test result details
If you click on one of the tests within the test report, you will find details about the execution and its results.
It will give showcase test step snapshots to understand what the execution did exactly. In the steps tab you can see all the interactions Octomind executed for this test run. Each step can be clicked on to see the exact screenshot at the time of execution.
failed test result detail, auto-playing all snapshots taken during execution, 08/2024
snapshots in test reports showing what happened during the test run, 7/2024
The failure state indicates that your app was not working correctly when executing the test. We will help you understand what when wrong and to debug either the app or your test itself if a test is red.
Debugging a failed test
In the debug tab you can see advanced options to find the issues in your app. You will find a link to a recording in Playwright Trace Viewer and the code for local debugging.
Test result debug tab, 08/2024
The first step when debugging should be inspecting the execution using the inspect in Trace Viewer button. A trace is similar to a recording, but has a lot additional information that come in handy when debugging. It allows you to quickly understand the test execution and the results of each action taken.
Example of a broken test reflecting a change in user flow, trace viewer screenshot 7/2024
To run your test on your own machine against your local dev environemnt use the run locally button. Find out how to run tests locally and debug using our open source Debugtopus.