Test reports
Test reports sum up the results of a test run - a single execution of all your active test cases.
Where to view your test reports
You will find a summary of the most recent 4 test runs in the test reports section in the project overview. Click on show more to see more than the last 4 test runs. The test reports menu in the sidebar lists all recent reports in chronological order, starting from most recent at the top.
Test reports are tied to a single target execution URL, so one specific deployment of your website or app, e.g. a branch deployment during a pull request.
'Test report page with test report list, 05/2025
Scheduled test runs
If you want to run your tests on a fixed schedule, the test reports section in the project overview gives you the option of a daily, weekly or bi-weekly interval.
This ensures that your app works flawlessly over time. We will send you an email notification if any of your tests fail.
Scheduling test runs in the Octomind app, 05/2025
Test reports in your CI/CD pipeline
If you integrated Octomind into your CI pipeline, we will comment test results back in your pull request after a completed test run.
Example of Octomind test results in a commit comment, 09/2023
There will be a row for each test with the following information:
| column | description |
|---|---|
| description | name of the test case |
| status ✅ | execution successful |
| status ⌛️ | execution in progress |
| status ❌ | execution failed |
| status 💔 | something went wrong on our side during execution |
| details | link which takes you to the test case details (see below) |
Test report details
When you click on a specific test report in the app or on the link in your commit comment, a test report overview with all the results and more details will open.
If triggered manually, you will find the user who triggered it and when. When triggered via CI integration, you will find a link to the commit that triggered it.
Example of a Octomind test report, 05/2025
Test result details
If you click on one of the tests within the test report, you will find details about the execution and its results.
It will show the test step timeline to understand what the execution did exactly. You can swipe through the steps back and forth to see all the interactions Octomind executed for this test run.
timeline of a broken test in test result detail showing what happened during the test run, 5/2025
The test result detail gives you insight into pre-classified root cause of a test failure, agent commentary, original prompt, you can compare test runs, check error log and more.
It gives you all the necessary info to inspect broken test and fix the app or your test, depending on what caused it. This is how you do it.
One view with all information to inspect what went wrong with a broken test, 4/2025
Debugging a failed test
In the maintain tests section of this documentation, we explain why tests fail, how you inspect the cause and how you can fix broken tests.