Create custom test case
Add more test cases to your test suite. Use AI generation or recording.
Prompt our AI agent to generate new tests
If you want to translate a custom user flow into a test, you can have it AI generated from your prompt. Give us a short prompt and we deploy our AI agent
to find the interactions leading towards your desired user flow.
Create new test case by prompting the AI agent, screenshot 07/2024
While our AI agent
generates the steps of your test cases, it informs you about its progress.
AI agent informs about its progress, screenshot 07/2024
If the AI agent generated wrong steps or signals a failed step with a yellow alert, you can:
- Restart the AI generation process by clicking
regenerate steps
. Select the last step you want to keep. All subsequent steps will be replaced with the new AI agent output. - Try a different prompt and restart the AI generation process by clicking
regenerate steps
. - Add, edit and remove steps manually. Our virtual locator picker helps you edit tests in no time. Learn how to edit test steps.
A generated test step failed, highlighted by yellow alert, screenshot 07/2024
Pre-prompted login test
If your app has a login functionality that wasn’t AI auto-generated during set-up, we have a pre-prompted login test case for you. You can trigger it after giving us test user credentials.
Go to settings
a.k.a. the gear icon
in the upper right corner and select test account
. Add test user credentials we can use to run the login test.
This test is also created on the fly for a new project, if we detect a login flow and you provide us with test user credentials.
Test user credentials for login test, screenshot 07/2024
Chaining tests
A user flow is virtually a chain of test cases. When using the AI agent, you can use a dependency to chain test cases together. Shorter flows are faster, more specific and more reliable.
We fill in some dependencies for you set-up Octomind. These are the cookies banner test (in case your site has a cookie banner) and required login test if you need to be logged in to operate the app. You can keep, remove or add other dependencies if you wish.
This is how you do it:
- Give us a path where the AI agent should start its test case generation process, e.g.
/products/
- Select a test case that should run before the AI agent is about to run.
Test case chaining, screenshot 07/2024
Free prompting best practices
-
Keep prompts short and to the point.
-
Mention all steps required, e.g. if you have to click away a cookie banner, mention it.
- Example without a cookie banner:
sign in using username $OCTO_USERNAME and password $OCTO_PASSWORD
- Example with a cookie banner:
accept cookies then sign in using username $OCTO_USERNAME and password $OCTO_PASSWORD
- Example without a cookie banner:
-
You have access to the following template variables that we will fill for you, so just use them in the prompt as they appear here and in the example above. (You can also use these when manually editing a step).
$OCTO_USERNAME
: The username you configured for use with your page in the sign up flow or the settings$OCTO_PASSWORD
: The password you configured for use with your page in the sign up flow or in the settings$OCTO_URL
: The URL of your page$OCTO_STABLE_UUID
: A random combination of letters and numbers that will be consistent through multiple references across a single run, but different if you run the agent again
-
These are the limitations so far:
- We can only handle flows which happen inside the app. Cross-app or cross-device flows are not supported, e.g. we do not cover email confirmation flow or a 2FA right now.
- We cannot handle captchas yet.
- Our AI agent might get blocked by robot detection on some high traffic sites. Sites / apps in production are more bot-protected than staging / test systems.
Record a test case
For more manual control and for cases that the AI model struggles with, we are offering the option to enter code directly. Ideally, produce your code with Playwright Codegen. You can find the commands required to use Playwright Codegen directly in Octomind, ready to be copy and pasted.
Once you finished your recording, just copy and paste the whole code into the code field.
Test case recording, screenshot 02/2024
Good to know
- Make sure to only record actions you actually want in your test case.
- Make sure to include all steps. E.g. the test runner might be located in a different geography than you and therefore might see different presets than you.
- We don’t support all interactions, yet. Clicks, entering text and visibility assertions are supported.