New UI, hydration issues and the winner of the tabs vs. toggle button bet

Hope you're doing well! I'd like to share the new batch of updates. It took a bit longer than usual, but it was worth it. We had our quarterly team get-together and decided to make room for more brainstorming rather than just crunching code.

The Good

• Agent benchmark automation. Every major change in the codebase impacts the AI agent performance. We used to check it manually. I know 🤯. Now, we get an automated ping whether the agent does what it’s supposed to.
• Shadow DOM & iFrames. I don’t want to jinx it, but it looks like we figured out how to handle them. Until you serve us the next 100 edge cases. 🎢
• Less is more. We reduced the number of discovered test cases but increased their relevance. We’ll focus on discovering MORE of the MORE relevant test cases next. 📈 
• Variable templates. We fill more robust variable templates into the test when your test steps are auto-generated. 🔡
• Full size side panel. No need to squint. No more neck pain. We’ve put important things to the center of your 💻 .
• Tabs are out! Tests in drafts, published, archived … looking for a specific test sent you on a ride through confusing lists and state tabs. Not anymore! Toggle to run the test with your test suite. 🔛 📴

‍

The Bad

Hydration issues in server-side rendering happen when the client-side javascript tries to take over the server-rendered HTML. They often go unnoticed until you want to do automated end-to-end testing. Which is a problem for us. Our users sometimes submit URLs with hydration issues.

We will figure out how to work around them. We prefer to solve the problem instead of saying 'you don’t comply with the specs'. 

Another diversion of the last two weeks - we tried to migrate our Open AI infra to Microsoft Azure to reap some commercial benefits. What could go wrong? They are buddies, right? Sure they have it all figured out. Well, we couldn’t. We didn't want to throw more engineering time out of the window so we aborted the mission. Another blog is brewing.

‍

The Complicated

App snapshots are complicated to handle. When editing a test case or looking for a locator, you need us to show you the correct one - the most up to date version of the relevant step in your app. Now, imagine you drag and drop steps tied to a specific snapshot to a different place in the test step sequence. We know that we don’t always get it right. We are working on it.

‍

The Highlight

Your project homepage has a new overview layout. So does the individual test case view. This is more than just UI fine-tuning. We tried to combine a focused design that is less cluttered while keeping the flexibility and all features at hand. 

First batch of UX testers had an improved experience. We'd like to hear from you. Go into the app and let us know what you think. Any room for improvement?

‍

---

‍

Somewhat related to the product - remember us getting rid of LangChain a few weeks ago? We wrote about it and it went viral. The reaction was overwhelming, we’ve obviously hit a nerve with the topic. See the discussion on Hacker News here or on Reddit here and here.  

‍

Thank you for sticking with us. If you have any questions or feedback for me, I’m listening.

‍