Avoid flaky end-to-end tests due to poorly hydrated Frontends with Playwright's toPass()

In this video we'll dive into the world of flaky tests in Playwright and synthetic monitoring with Checkly. We examine a site with poor Frontend hydration patterns, their effect on test stability, and how to work around them.
Learn how to avoid using artificial delays and implementing a retry mechanism with Playwright's 'toPass()' method to achieve stable testing instead.
0:00 Intro
0:38 Example of a poorly hydrated page
2:17 Recommended fix for poor hydration
3:32 Custom action retries with 'toPass()'
5:18 'toPass()' options
6:12 Running Playwright using the `--repeat-each` option
6:43 Outro
Checkly Synthetic Monitoring: www.checklyhq.com/product/syn...
Playwright 'toPass()' docs: playwright.dev/docs/test-asse...
#playwright #testing #monitoring #syntheticmonitoring

[@sayyedumair1017]

Great video. This is really helpful in my daily work. Thanks for sharing this.

[@roblesrt]

This appears to be a better workaround than Webdriverio where I struggle with. This is very helpful in my case. Thanks for sharing!

[@ChecklyHQ]

Yeah, testing poor hydration is tricky, but this turned out to be valuable for me. Happy the video was helpful!

[@_firearmy]

Great video and explained very clean. Thank you!

[@ChecklyHQ]

Glad it was helpful!

[@andreaskarz]

That's really cool - I love Playwright 😍😍

[@og4789]

Creepy I exactly faced this poor hydration behavior on the tested web app few days ago for the first time. What a timing, Thank you for showing that option! 🙏

[@ChecklyHQ]

Great! 🎉 Happy this video came at the right time. Let us know how it goes. ;)

[@lfc67671]

To further this incredible timing I literally ran into this issue today! Great vid thank you Checkly!

[@ChecklyHQ]

@@lfc67671Yay! 🎉

[@harshasuraweera]

great explanation!

[@ChecklyHQ]

Glad it was helpful! 💙

[@dvdpeeters]

Super nice !!

[@ChecklyHQ]

Thanks. Happy it's been valuable. 🎉

[@HossamEldinMohamed3]

Thanks for the awesome video and tips! Keep up the good work

[@ChecklyHQ]

Thanks for the kind words. Happy the videos are valuable. :)

[@barscanates5573]

Hello! I really appreciate that. Could you please create a video list focused solely on fighting flakiness in testing? This is a critical area for all testers.

[@ChecklyHQ]

This is a great idea and I'll come to this in the upcoming weeks. I'll let you know here.

[@coderkind]

Another great video; thank you. I’d possibly disagree with the “right fix” advice on Playwright’s docs to disable interactive controls until after hydration though, as that could cause confusion and frustration to screen reader users. They might attempt to interact with an element initially announced as being disabled/dimmed, and when that control was subsequently enabled via hydration then you’d have to manage how to update that state to the screen reader user.

[@ChecklyHQ]

Love this comment! You make a great point. I followed the PWT docs for the sake of the storyline of the video. But I think you're correct. Disabled controls aren't focusable and won't trigger validations - and generally in the UX/a11y community it's advised to not disable controls for these reasons. But yeah, the topic of poor hydration + disabled controls is probably for another day and/or video. Thanks! 💙

[@coderkind]

@@ChecklyHQthank you. It was more of an end-user issue I was highlighting (not necessarily Playwright operation); that it’s not great to set out a page as having x elements non-operable and to then amend that, as it’s something that then possibly has to be managed so the screen reader user is aware they can use those elements. It might be edge-case; that the page will hydrate and elements are enabled before a screen reader user attempts to use those elements, but it might be more of-a-thing in things like modals (where you often have a button default-focused).

[@aaronmcadam]

Fantastic video! I'm curious: how did you create the demo app to force a hydration error for your test to fall?

[@ChecklyHQ]

Hah - that's a great question! For this example I only delayed adding the event listeners via a `setTimeout`. 🫣 But the same problem could appear because of various reasons. :)

[@maverick53500]

Unfortunately this solution works 50% of the time. I used it in 2 tests, in one of them it works, and in the other it doesn't. I have 2 assertions inside (toBeVisible and toHaveText) followed by page reload. The 'page reload' part is not executed for some reason... I had to wrap it in try-catch block, with 'expect' part inside 'try', and the page reload inside 'catch'

[@ChecklyHQ]

Happy you figured it out. :)

[@nickfwhite4296]

I am new in test automation, and I needed to test redirects after page reload, so I used await page.waitForLoadState('load'); await page.waitForTimeout(500); await page.reload(); await page.waitForLoadState('load'); await page.waitForTimeout(500); await expect(page.locator('.product-card-wrapper').first()).toContainText('PLN'); but .toPass() is much faster and stable, thank you a lot

[@ChecklyHQ]

Yay! Happy the video has been valuable. 💙