​
Prompt our AI agent to generate new tests

The fastest way to create a new test case is to have it AI generated. You give us a short prompt and we deploy our AI agent to find the interactions leading towards a desired user flow.

AI agent informs about its progress

Create new test case by prompting the AI agent, screenshot 07/2024

While our AI agent generates the steps of your test cases, it informs you about its progress.

AI agent informs about its progress as it generates steps

AI agent informs about its progress, screenshot 07/2024

If the AI agent generated wrong steps or signals a failed step with a yellow alert, you can:

  1. Restart the AI generation process. Select the last step you want to keep. All subsequent steps will be replaced with the new AI agent output.
  2. Try a different prompt and restart the AI generation process.
  3. Add, edit and remove steps manually. Our virtual locator picker helps you edit tests in no time. Learn how to edit test steps.
A generated step failed and shows a yellow alert

A generated test step failed, hightlighted by yellow alert, screenshot 07/2024

​
Pre-prompted login test

If your app has a login functionality that wasn’t AI discovered at the set-up, we have a pre-prompted login test case for you. You can trigger it after giving us test user credentials.

Go to settings a.k.a. the gear icon in the upper right corner and select test account. Add test user credentials we can use to run the login test.

This test is also created on the fly for a new project, if we detect a login flow and you provide us with test user credentials.

Login test user variables for login test

Test user credentials for login test, screenshot 07/2024

​
Chaining tests

A user flow is virtually a chain of test cases. When using the AI agent, you can use a dependency to chain test cases together. Shorter flows are faster, more specific and more reliable.

We fill in some dependencies for you set-up Octomind. They are the cookies banner test if cookie banners are used and required login test if you need to be logged in to operate the app. You can keep, remove or add other dependencies if you wish.

This is how you do it:

  • Give us a path where the AI agent should start its test case generation process, e.g. /products/
  • Select a test case that should run before the AI agent is about to run.
Test case chaining

Test case chaining, screenshot 07/2024

​
Free prompting best practices

  1. Keep prompts short and to the point.

  2. Mention all steps required, e.g. if you have to click away a cookie banner, mention it.

    • Example without a cookie banner: sign in using username $OCTO_USERNAME and password $OCTO_PASSWORD
    • Example with a cookie banner: accept cookies then sign in using username $OCTO_USERNAME and password $OCTO_PASSWORD

  3. You have access to the following template variables that we will fill for you, so just use them in the prompt as they appear here and in the example above. (You can also use these when manually editing a step).

    • $OCTO_USERNAME: The username you configured for use with your page in the sign up flow or the settings
    • $OCTO_PASSWORD: The password you configured for use with your page in the sign up flow or in the settings
    • $OCTO_URL: The URL of your page
    • $OCTO_STABLE_UUID: A random combination of letters and numbers that will be consistent through multiple references across a single run, but different if you run the agent again

  4. These are the limitations so far:

    • We can only handle flows which happen inside the app. Cross-app or cross-device flows are not supported, e.g. we do not cover email confirmation flow or a 2FA right now.
    • We cannot handle captchas yet.
    • Our AI agent might get blocked by robot detection on some high traffic sites. Sites / apps in production are more bot-protected than staging / test systems.

​
Activate your test case

Check if your test is on. Use the toggle button to switch the test to “on”. This means the test is active - it was added to your active test suite and will run when a test report is triggered.

Toggle button to activate test

Toggle button to activate your test case, screenshot 07/2024

​
Record a test case

For more manual control and for cases that the AI model struggles with, we are offering the option to enter code directly. Ideally, produce your code with Playwright Codegen. You can find the commands required to use Playwright Codegen directly in Octomind, ready to be copy and pasted.

Once you finished your recording, just copy and paste the whole code into the code field.

Test case recording

Test case recording, screenshot 02/2024

Good to know

  • Make sure to only record actions you actually want in your test case.
  • Make sure to include all steps. E.g. the test runner might be located in a different geography than you and therefore might see different presets than you.
  • We don’t support all interactions, yet. Clicks, entering text and visibility assertions are supported.
First stepsEdit a test case