Prompt AI Agent
Use natural language to prompt our AI agent to generate new tests
Prompt the Agent to generate a specific test case
If you want to translate a custom user flow into a test, you can have AI generate it from your prompt. Give us a short prompt and we deploy our AI agent to find the interactions leading towards your desired user flow.
Create a new test case by prompting the AI agent, 07/2024
While our AI Agent generates the steps of your test cases, it informs you about its progress.
AI Agent informs about the progress of test step generation, 07/2024
If the AI agent signals a failed step with a yellow alert, click on repair test to help the Agent finish the generation successfully.
The AI agent needs help your input to finalize generation - click repair test, 05/2025
Follow the instruction of the agent to repair the test.
Follow the instructions of the AI agent to repair and finalize test generation, 05/2025
Pre-prompted login test
If your app has a login functionality that wasn’t AI auto-generated during set-up, we have a pre-prompted login test case for you. You can trigger it after giving us test user credentials.
Go to the environments menu item in the left project menu bar and click on the default environment in the list.
Select the authentication tab and add test user credentials we can use to generate the login test.
This test is also created on the fly for a new project, if we detect a login flow and you provide us with test user credentials.
Provide test user credentials for login test generation in environment settings, 05/2025
Add test user credentials for the login test generation in the authentication tab of the environments settings, 05/2025
Or check out this video instead:
Chain tests for faster test runs & maintenance
A user flow is a chain of shorter test cases. When using the AI agent, you can use a dependency to chain test cases together. Shorter flows are faster, more specific and more reliable.
We fill in some dependencies for you at set-up of your Octomind project. These are the cookie banner test (in case your site has a cookie banner) and required login test if you need to be logged in to operate the app. You can keep, remove or add other dependencies if you wish.
This is how you do it:
- Give us a path where the AI agent should start its test case generation process, e.g.
/products/ - Select a test case that should run before the test AI agent starts its generation.
Chain tests in dependencies when you create them, 08/2024
Free prompting best practices
-
Keep prompts short and to the point.
-
Mention all steps required, e.g. if you have to click away a cookie banner, mention it.
- Example without a cookie banner:
sign in using username $OCTO_USERNAME and password $OCTO_PASSWORD - Example with a cookie banner:
accept cookies then sign in using username $OCTO_USERNAME and password $OCTO_PASSWORD
- Example without a cookie banner:
-
You have access to pre-defined variables that we will fill for you, so just use them in the prompt as they appear here and in the example above. You can also use these when manually editing a step.
-
These are the limitations so far:
- We can only handle flows which happen inside the app and in certain cases navigate to different urls. Cross-device flows are not supported. We are able to test email confirmation flow with an Octomind provided email address.
- We cannot handle captchas yet, unless it’s avoidable via custom headers
- Our AI Agent might get blocked by robot detection on some high traffic sites. Sites / apps in production are more bot-protected than staging / test systems.
- Our Agent is blocked by a rate limiter. This is how you mitigate rate limitation.
- The Agent might not support the required type of interactions & assertions yet. Manual intervention will be necessary.
- Timing and hydration issues can cause the Agent to fail, too.