How to Speed up your Playwright Tests with shared "storageState"

[@ChecklyHQ]

Quick comment after releasing this video: you can and should only reuse session / login data when your tests aren't modifying the existing user. Otherwise, your tests will conflict with each other. In these scenarios, it's better to create a new user for each tests.

[@suda-nz]

Your tips have everything I was looking for, thank you so much. I wouldn't have started my test automation project without your channel.

[@ChecklyHQ]

@@suda-nz thank you! Happy the videos are valuable. 🦝

[@dva_kompota]

Neat way to reuse logged-in state, when you don't create unique users for each test and can't (or don't want to) auth via API calls! Great content! Thanks 👍

[@phatle4855]

I cannot imagine that this amazing concept exists! Thanks sir.

[@FolamiMilorad]

this guide is fantastic! just want to ask how can we implement it in CircleCI for our test environments

[@MarkusGarmeister]

Hey Stefan, I really love your videos - they've helped me a lot! I’m currently facing an issue while testing a webpage (Web3) that interacts with the blockchain. I need to connect to a wallet before running my tests. I followed your approach of setting up the wallet connection through a setup file, as you explained in the video, but unfortunately, it doesn’t seem to be working. I think the connection isn't getting stored properly in the JSON file. Any advice on what might be going wrong? Thanks a lot!

[@MarkusGarmeister]

Just a quick follow-up, Stefan - I was also wondering if you could consider making some videos about testing with extensions and Web3 in general? There's really not much material out there on this topic, and it would be super helpful to get your insights on it. Thanks again for all the great content!

[@ChecklyHQ]

@MarkusGarmeister Thanks for the kind words! 🦝 Unfortunately, I don't have any Web3 experience so I doubt we'll cover these topics any time soon. :/

[@PakoVere]

Thanks for great video. I have already had API authentication in place that bypassed login of every single test. So I tried your storageState approach and did very basic measuring between those two approaches. I ran a single suite of 10 tests, with 3 workers on my workstation. I did 10 repeats and calculated average run time. API authentication in before each: Average run time: 14486.2 ms Setup with storageState: Average run time: 17121.5 ms So the storageState is on average slower by 2,635.3 ms. The setup itself seems to be on average ~ 1,700 ms This means that while each steps executes less amount of code, I guess the setting the of the storageState for each test takes a bit of time. And rough comparison would be they run by 0.1 s longer per test. I would judge that negligible. So to sum it up. It seems storageState does not provide any performance advantage over calling auth API and setting auth tokens before every test. But storageState definitely reduces the amount of code that is in your test file and improves readability. I would be interesting hearing your experience or anyone else on this topic. :)

[@ChecklyHQ]

@PakoVere This is great! 💙 Thank you so much for this investigation, this is super interesting. Let me summarize if I got this correctly. In your benchmark, you only apply the already created storage state (`setup` was already done?) and this approach is slower than making an api call and setting the value to your test runs? If so, this is interesting and I woudln't have expected this outcome. Could you share slimed down example code somewhere? I'd love to look into it! That said, I think it's a great investigation already, but in my experience sometimes logins need a browser and authenticating via API isn't always possible. But of course, if your API approach works and is faster - that's great stuff! 🎉

[@stephena8965]

Great video, everything worked like a charm! Quick question, could you theoretically use similar logic to store login state when running the scraper in a production environment with Playwright Library?

[@ChecklyHQ]

I haven't tested it, but it should work the same when using the Playwright lib. :) The only thing to keep in mind then is that you'll have to run the setup script (it's done via projects in the video) somehow yourself. :)

[@stephena8965]

@@ChecklyHQ Amazing, thank you so much for the info and quick response! I’ll report back when I cross this bridge 🙏🏾

[@ChecklyHQ]

@@stephena8965 Looking forward to hear back!

[@stephena8965]

@@ChecklyHQ I can confirm this works! I took the following steps: 1. Created an async function called login.js that returns the cookies, or sets and returns them if needs be - it's important to make sure this is just a normal async function, not a self invoking one 2. Created an async getCookies function to handle the cookie logic for cleanliness. It's just a try catch statement that either returns the cookies array or an empty array 3. Call the async login function in my index.js file on each run, so now it's able to handle the cookie logic depending on whether the cookies array is empty or has length Perhaps there's a dryer way to do this with browser contexts etc but I'm a little new to this. Everything seems to be working fine tho so I'll optimise at a later date! Thanks again for the tutorial-couldn't have reached here otherwise!

[@glebmirosnikovs6722]

Thanks man, cool tip! You said we can remove tick from setup, so auth step is not running before each test. But, what happens, when tests are running remotely in pipeline? How could I make setup test to run only once, and then take all info from storatestate? For example, I have 100 tests. I need setup step to run only before first step, not before all of them.

[@ChecklyHQ]

Thank you! 💙 Yes, what I meant in the video is that during development you can run the setup step once and than reuse the storageState for as long as it's valid. When running things in a CI pipeline Playwright should figure out everything for you automatically. When you run `npx playwright test` it'll figure out the project dependencies and run `setup` for each worker. As long as all your tests run in the same worker, `setup` should only run once. If you're parallizing things (which you most likely do), `setup` will run for each worker because workers can potentially run in different invironments. I hope this makes sense. :)

[@glebmirosnikovs6722]

@@ChecklyHQ thanks for your reply

[@Jitowix]

By default we have checkly.config.ts, not playwright.config.ts right? We just need to replace it?

[@ChecklyHQ]

Great question! The `checkly.config` isn't a default replacement for your Playwright configuration because additionally it allows you to configure your Checkly monitoring infrastructure. If you want to reuse or define values in your `playwright.config` you can do that, too. :) www.checklyhq.com/docs/browser-checks/playwright-test/#global-configuration I hope this helps. 🦝

[@maniladevotee3595]

This is useful for login with MFA functionalities.

[@benjsoft]

Hi, thanks for this very useful tip to increase the speed of running tests in Playwright. One issue though I encountered is that if we have several tests and after 20 minutes when running other tests, it failed because the token already expired. Do you have any best practice or recommendation in alleviating this issue?

[@ChecklyHQ]

Thanks for the question. Hmm... This is an interesting case. How many tests are we talking about here? I'd look into ways to speed up your test first, because 20min seems like a reasonable long time for an authentication token. Could you parallelize your tests more so that more things run to reuse the token and browser state?

[@benjsoft]

@ChecklyHQ 98 tests to be exact, its running in 4 workers already

[@ChecklyHQ]

Assuming that you're running the tests sequentially in every worker (i think this is recommended in the docs somewhere? 🤔) you could parallelize even more to decrease the overall time? If that doesn't work, unfortunately, I can't give any better advice because "refreshing storage state" seems wrong and is also missing the point of the whole concept. 😅

[@benjsoft]

This is great tip! Thanks!

[@lemontec]

7:46 this step is confusing because I’m not using the UI when configuring the CI tests right? Is this only required with the UI and otherwise tests respect the dependency array? Or would this also run setup every time before the each test?

[@ChecklyHQ]

UI mode is generally only for development. :) When you run the tests in CI or via the command line the dependency array will be respected either way and setup will run for each executed worker.

[@Wakkyguy]

I am working on an application where the login flow is not based on the user's input. Instead the user is redirected to an Okta auth page which internally uses kerberos and eventually redirects the user back to the application. The redirect logic and local storage update is handled by Okta auth js and okta react packages. In this scenario, how can I stub my auth state to a temporary json file? When I try to run my tests, the auth flow breaks as Okta server doesn't recognize the user when the application is running via playwright's browser.

[@ChecklyHQ]

Heyoooo. 👋 Unfortunately, I haven't tested this flow yet and neither do I have a quick test setup to play around with it. So, unfortunately, I can't give guidance here. :/

[@vaveal3220]

Спасибо за видео и время которое вы потратили на его создание. Вы очень помогли

[@MrAnujbajaj]

Is there a workaround if your app is integrated with OKTA authentication?

[@ChecklyHQ]

Unfortunately neither have I done OKTA testing nor do I have an OKTA setup available. So generally, I can't give advice here right now. But if you make it to pass the OKTA 2fa flow storage state should still work. :)

[@MuraliRajamanickam]

It works until my session is active i .e.if the maximum time for a session is 15 mins . If I try to access my stored. Json file after 15 mins it would show session time out . How to resolve it

[@ChecklyHQ]

I'm not sure if I understand your question correctly, but the session storage to be invalid at some point is expected. In CI/CD it would always run before your tests so it still speeds up your tests. In development you'd have to rerun `setup`.

[@phamquynh811]

It' great. Could you make a video to guideline "Testing multiple roles with POM fixtures", please?

[@ChecklyHQ]

Could you explain a bit more? Are you asking for an example showing how to have e.g. an admin/user test case? What issues are you facing?

[@phamquynh811]

@@ChecklyHQ I have test cases need to be interacted between user and admin in one test. This is how I set it: const adminContext = await browser.newContext({ storageState: '.auth/admin.json' }) const adminPage = await adminContext.newPage() const userContext = await browser.newContext({ storageState: '.auth/user.json' }) const userPage = await userContext.newPage() await adminPage.goto(commonData.stgUrl) await userPage.goto(commonData.stgUrl) But I need to paste it on each test, that is my issue. Hope that you give a suggestion

[@youssefmrabet4837]

Thanks for the tips!

[@DJ_Locks]

Nice trick, now do it with session storage :3

[@ChecklyHQ]

Haha, while possible with page.evaluate() i'll leave that for another time. 🫣

[@knivesoutbitch]

@@ChecklyHQ ugh i need this!! was bummed when i only saw localStorage being saved

[@Uradha1]

I can create the storage using setup, but it's not passing to next test. In the browser, it was showing" about:blank". But on your video pass to another test.

[@ChecklyHQ]

Without seeing your code unfortunately it's impossible to help. A link to the code is in the description.

[@Uradha1]

I have 7 tests to run, after using this, first, it's taking only 4 workers instead of 7 workers, and after that, the other 3 workers started. Do I need to change anything?

[@ChecklyHQ]

What are you `fullyParallel` and `workers` settings? By default, PWT tried to parallelize as much as possible. The `dependency` setting shown in this video only controls the order of projects run. :)

[@abhishekshukla2745]

I did the exact setup but on next load it again lands on login page.

[@ChecklyHQ]

I'm sorry to hear that. Without additional info and example code, it's impossible to help, though. You can check the working example on GitHub: github.com/checkly/playwright-examples/

[@santiagoquintana2233]

Thank you, sir!

[@brandonzafiro8449]

How can we handle this on CICD ?

[@ChecklyHQ]

Heyoo. :) Can you expand the question a bit more? This approach should work in CICD without a problem. :)

[@brandonzafiro8449]

@@ChecklyHQ Thank you for your response! I wanted to clarify-should we include that type of information in the repo as well? Additionally, what would be the best approach to manage credentials securely when using Playwright in a CI/CD pipeline? I’d appreciate any suggestions or best practices you could share. Thanks again!

[@ChecklyHQ]

@@brandonzafiro8449 Sure thing. :) No, you wouldn't put these credential files in your repo and they should live in gitignore. (they'll also expire eventually) In CI/CD, you run the setup step as shown, generate a new credentials file, and your tests will read it to set the state. This way, everything is always fresh and you don't need to check in additional files. I hope this makes sense. :)

[@Dzidziolino]

Just worth to mention that this method is only viable when you do not change any data on a user or in the system that is shared between tests. In most cases you need to create new user and log in via api.

[@ChecklyHQ]

That's a great call thank you! And for sure, it depends on the tests and what they perform!

[@ChecklyHQ]

I made a comment and pinned it. Thanks again for this valuable comment!

[@iHorus]

Did someone try it with auth0, does it work fine? I loose login after a 2-4 tests... UPDATE: works fine, it's just logout of one of the tests cancelled the state (what a surprise) No logout - runs fine

[@ChecklyHQ]

Great to hear you figured it out!