Test steps debugging

Previously, we’ve introduced the most common reasons why Octomind’s tests fail and showed how to fix them if the prompt is causing the AI agent getting stuck. In this debugging guide we’ll give you a ‘how to’ for debugging your test step by step.

This is where we’ll be diving deeper into Octomind’s debug tools, starting with the visual editor panel.

​

Visual locator picker

Steps are the translation of the prompt into a list of interactions or assertions that target a single specific element known as the locator - Playwright’s API for locating elements.

Interactions simulate user actions by sending events to the browser:

• Enter text
• Press key
• Click
• Selection option
• Hover

While assertions inspect the state of the page:

• Visible
• Hidden
• Disabled
• Have value

The current locator for the step is highlighted in green with the visual locator picker hovering next to it.

The change locator button is used to set the element the interaction of the assertion is performed on. What’s cool is that the web page preview is not a static screenshot, but a fully scrollable version of the page, making it easy to select a new locator.

When an element is clicked, the Agent chooses the appropriate Playwright locator code which is displayed in the current locator editor. Should you want to customize the locator, the editor puts Playwright’s full locator API at your disposal, with code-completion, syntax checking, and validation to check your locator code returns an element.

​

Step failure debugging

A step will fail if the locator isn’t found within the 30s timeout period or if the assertion is not met (e.g. expected text not found).

Let’s see how to fix locator and assertion errors using a broken ‘search for a product’ test case.

The first step has a locator error caused by the search placeholder text—expecting ‘Search…’, not ‘Search here…’

The select locator button is used to select the search field which automatically updates the placeholder text.

Now that we’ve fixed the locator failure, let’s move onto fixing the assertion failure which expects an element containing the text ‘Showing 961 products’ to be visible.

We’ll change the locator to be something stable by selecting the ‘Search in…’ text, but we’ll manually edit the text property to remove the duration as it bound to be different between test runs.

That’s how easy it is to fix locator and assertion errors using the visual locator picker. But if you want to go deeper, Octomind has you covered.

​

Code-level debugging

If you’re a developer, you can also debug the underlying Playwright testcode the Agent has generated by clicking the debug tab.

To run the test locally, copy and paste the code provided, or you can click the run locally button to generate the Debugtopus CLI command for debugging the test on your local machine.

Now that you’re up to speed on how to fix test failures from the test case design page, let’s look at how to diagnose and debug failing tests from a test report.