Extract Multiple Web Pages with Power Automate Desktop - Full Tutorial

This video tutorial shows how to extract multiple web pages with Microsoft Power Automate Desktop. You'll learn how to create a loop with a dynamic selector and ensure you don't overwrite Excel data.
🌍 The page I scrape in the video: portal.cfm.org.br/busca-medicos
VIDEOS FOR YOU:
🔹 Web Scraping in Power Automate Desktop (1 HOUR): • Web Scraping in Power ...
🔹 Save Web Data to Excel - Advanced PAD Use Case (2 HOURS): • Save Web Data to Excel...
POWER AUTOMATE DESKTOP GROUPS:
👨‍👩‍👧‍👦 Discord: / discord
- Join my network with 4200+ Automation/RPA developers, where we solve Automation/RPA problems, network and help each other upgrade our careers. It's free!
👩🏻‍🎓 LinkedIn: / 12566435
👨🏻‍💻 Facebook: / 754059285247921
FOLLOW ME:
💼 LinkedIn: / andersjensenorg
📸 Instagram: / andersjensenorg
👨 Facebook: / andersjensenorg
🐦 Twitter: / andersjensenorg
💌 Email Newsletter: andersjensen.org/email-newsle...
0:00 Web Scrape a Single Page
Power Automate Desktop is next to none regarding data extraction of web pages with no code. Directly apply Regular Expressions to the results if you want to filter them further.
3:13 Multi-page Web Scraping
We have search results on multiple pages. In addition, there is no next button, only page numbers. So the solution is to create a loop with a dynamic selector - both dependent on the current and the total page number.
6:37 Creating a Dynamic Selector
We create a dynamic selector in Power Automate Desktop so that we can click all the next page buttons.
12:48 Write to Excel (multiple pages)
We'll ensure we don't overwrite Excel data when continuously writing newly extracted data to Excel.
#powerautomate #powerplatform #office365

[@andersjensenorg]

Thanks for all your questions and comments 🥰 I've created a Discord, where we can help each other with RPA and network around our careers: discord.gg/iloveautomation

[@wallyrogers7501]

I joined. Easy to get lost in there. Need an AI to find the right topic.

[@VitorReis-rn8ho]

I have struggled a lot, but, thanks to your instructions, I managed to adapt small parts of the script to my needs. I already knew your work teaching autohotkey. Your teaching is fantastic!!! Thank you very much!

[@andersjensenorg]

Hey VitorReis, thanks a lot - Have you seen my discord (link in the description)? We're 8000 RPA developers helping each other, and it's of course free. Kind regards, Anders

[@glennganey6217]

Thanks for another great tutorial. The dynamic selector was elegant. I am still scratching my head over doing a google search for a refrigerator part (DA47-00322B). I go through the standard selector UI assignments for the results title and for the body of text, sometimes the part number will not appear in the title but does appear in the body of text. When I run PAD it picks up only nine of the thirteen results. Using your tip (F-12) I look at the code, but I am no web programmer, and do see a significant number of differences between some of the results. The object of the exercise is to write a URL & part number if the part number is explicitly found in either the title of body of text and of course save the results in excel. I do believe this would make for a good tutorial since so many people use google searches. Thanks for any incite you can shed.

[@sergiobarros5541]

Mr. Jensen, you rock! Thank you for this wonderful tutorial. Greetings from Brazil!

[@andersjensenorg]

My pleasure, thanks for the nice comment 🥰 Kind regards, Anders

[@kaushik9gohil]

I learned something new, was able implement in actual real life requirement. Thank you very much for making it so easy and simple.

[@andersjensenorg]

Hey @kaushik9gohil. Thanks a lot 😊 Have a great day. You're more than welcome at my Discord, where we're 8000+ RPA developers networking and solving problems: kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@adenilsonx.franco5287]

Thank you very much for the tutorial that prepared Mr. Anders Jensen, it will help me a lot, I will use it in other researches, because I can use your techniques.

[@andersjensenorg]

Thanks a lot, Adenilson 😊 I'm so happy, that I could help you - have a great day. Kind regards, Anders

[@thehspace]

Very helpful. Thank you!

[@andersjensenorg]

Hey @thehspace. Thanks a lot 😊 Have a great day. You're more than welcome at my Discord, where we're 8000+ RPA developers networking and solving problems: kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@Chime-In-CelebQuotes]

Thank you Anders !! this is very helpful :)

[@andersjensenorg]

You're very welcome, Neha Puri - your support helps me a lot 😍 You're more than welcome at my Discord, where we're 6000+ RPA developers networking and solving problems: kzbin.info/www/bejne/roipq2CJbpuLfdE

[@agamenon1953]

Thank you for such a great video Anders. I wonder if you can post a video explaining how to scrape web pages with infinite scroll.

[@andersjensenorg]

My pleasure 🙌😊

[@Hungry_kitchen777]

Hi Anders thanks for the great video, may I know how to scroll down and extract the entire table from web page in PAD, Iam trying but able to get only the one's displayed on the screen, hope to see the repl soon😊

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 4500+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@sujitbhowmick9517]

Love from India ❣️👌

[@andersjensenorg]

Thank you so much, Sujit 🥰

[@manjunathreddy4320]

Thanks for awsome content! Could not see advanced settings option in add ui element in the Power automate desktop, and I am using latest vestion.

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 10,000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@Palmakify]

Thank you for the video, always helpful. I have a question: When I try to extract the list of HRef from a website, I only get 2 results written into my excel, but on the preview extraction, it shows me more than 2 results. It's a bit strange that the preview results and the actual extracted results are different.

[@andersjensenorg]

Thanks a ton, CP. Can you link to the site? 😊

[@happy9955]

great teaching

[@andersjensenorg]

My pleasure ☀️🙌

[@syedimtiaz3558]

Hi Anders, Thanks for your video. What if you had to run an inner loop for each employee to get some info from his/her profile and then come to the root page which has pagination? Is there any workaround for a scenario this?

[@andersjensenorg]

My pleasure, Syed. It's almost the approach I take here (just with cars): kzbin.info4zmviNtOFwA Kind regards, Anders

[@suraj_singh321]

Kudos Anders Sir Make a video on Working with CMD Sessions operations from basic to advance in Power Automate Desktop

[@andersjensenorg]

Hey RPA_Enthusiast Thanks a lot for writing. I'm getting more than 50 questions/comments every day and having difficulty answering everyone - but I try my best. I've created my Discord, where we're 4300+ RPA Developers networking and solving problems. You're very welcome: andersjensen.org/#community Kind regards, Anders

[@missnlahi]

Thanks I'll try it. I think this addresses why my script was only working for the 1st page of data

[@andersjensenorg]

Nice to hear, please let me know if it worked 🙂

[@Taxita]

Thanks for doing this. I was wondering how to do that.

[@andersjensenorg]

My pleasure, Falcon 🙏👍

[@stanTrX]

Thanks. There is also set pager property for the next pages, does that work as well?

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 9000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@freeacademy701]

Great video, can power automate open multiple webpages or tabs at the same time and filling forms or performing clicks on every tab at the same time?

[@andersjensenorg]

Hey Free. Yes 100%, but not 100% at the same time, but right after each other. Kind regards, Anders

[@JoelNove]

Thank you for your video. I'm creating a flow that extracts my banking information and saves the data to a SharePoint list. I was curious, do I need to keep Power Automate desktop open on my laptop at all times or can the flow be triggered and run on the cloud while laptop is shutdown? Trying to avoid going python rout. Thanks!

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 8000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@choiboix]

Can you do a tutorial of scraping similar formatted tables from multiple PDF's to a excel data table? I know you did a powerAI one extracting fields and tables from PDF but that was kind of vague near the end and confusing with the coding stuff. Is there a simpler and more straight forward to set a flow?

[@andersjensenorg]

Hey Michael. Are they text or image PDFs? 😊

[@choiboix]

@@andersjensenorg Texts. I believe the PDF was converted from Excel as it has a table format like Excel.

[@dDse7en]

thank you very much for this video, but i have a question, why is your data in excel is separate nicely from value scrap from web into each individual cell, meanwhile mine is all the data scrap from the web is lump into a single cell

[@andersjensenorg]

Thanks a lot 👍😊 How does your DataTable with the scraped data look like? If one cell, you scraped wrong. You're more than welcome to post the problem on my discord :) It's 100% free and the link is in the video description. Kind regards, Anders

[@fanisz5331]

Mr Jensen thank you very much for you valuable support. I have a quick question about regex. If got the return value is lets say 1000,00 € and I want to retrieve only "1000", what regex should I type ?

[@andersjensenorg]

My pleasure, you can try with this: \d+(?=,) Kind regards, Anders

[@fanisz5331]

@@andersjensenorg Thank you so much for you response. I forgot to mention that the data has a . . It's 1.000,00€ . I noticed that this regex does remove data after comma but it also removes data before dot. It returns 000 Is there a regex that returns 1.000 ?

[@andersjensenorg]

@@fanisz5331 Oh, then you can try: [\d.]+(?=,) 🙂

[@fanisz5331]

@@andersjensenorg thank you so much! Works like a charm!!!

[@none1615]

after 30 seconds, I subscribed , What a wonderful channel and content , thanks for sharing with the community .. I am looking for a simple solution to copy all contents of OneDrive including Folders and subfolders and files beneath them and copy/move to sharepoint location .... That scenerio somehow not covered anywhere yet , all examples I've seen were based on file copy / move but none of them showing how to do deal with folder or subfolder creations .. If you read this message and think that would be valuable to the rest community and make short video , that would be greatly appreciated .:)

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 4900+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@suraj_singh321]

Hii Anders Sir make a seperate video on Regex what is Regex how we can build Regex and use in uipath and power Automate desktop

[@andersjensenorg]

Hey Suraj. I will do my best, but have you seen this: kzbin.info/www/bejne/ip7IeJSuoNili6M Kind regards, Anders

[@billiedwong6982]

great video and sharing! thanks! I tried to run multiple-pages based on your example, I got error: div[id="paginacao"]>div[class="paginationjs"]>div[class="paginationjs-pages"]>ul>li[data-num=16] or 6..not found. any advice?

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 9000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@Philbnyc]

I want to take a screenshot for websites from an excel url list , and save the screenshots in a directory with the url name ..possible ?

[@andersjensenorg]

Thanks a lot, @philborrero592 🤗 Your feedback helps me immensely. You're more than welcome at my Discord, where we're 11,000+ RPA developers networking and solving problems: kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@wallyrogers7501]

I did pretty good until Create the Dynamic Selector. My code in Inspect properties looked nothing like yours. It would be great to add on the Power Automate version of that step in addition to the developer tools.

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 12,000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@suraj_singh321]

Make a use case different challenging scenarios of Selectors to make Dynamic using power Automate Desktop

[@andersjensenorg]

That's a great idea, thanks a lot, Suraj!

[@matusklubica1256]

Hello im struggling with the third part of your video, i cant make it work, my source code looks different and I cant deal with it, i would really like to master it, thank you for your help, love your videos :)

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 11,000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@viveksview7901]

hi if we dont know the last page instead i could see next button on screen for mulitiple data extraction

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 5800+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@user-og2hy5nc9s]

Hi, i am trying to gather data from a website that has further links on the page. So i need to click on the link extract that data, go back click on the next and so on. There are multiple pages as well. I cant seem loop it through correctly. Can someone provide a step by step guide on how to do this i am new to power automate.

[@andersjensenorg]

Of course. Kindly see this: kzbin.info4zmviNtOFwA 😊 Anders

[@BrainUpp]

Also scrape data from inside from every link multiple data upto multiple pages

[@andersjensenorg]

Hey Brain Up. Please check this: kzbin.info/www/bejne/aqvQp5yEqbR5raM Kind regards, Anders

[@BrainUpp]

@@andersjensenorg yes i saw this you have extrated only one item i mean to say to scrape multiple items

[@hossameldinelsayed2689]

What if i need a scrapper to run everyday and extract html table, the thing is the page count changes every day, when i am trying the pager option it keeps triggering the next button and ends up with an error failed to extract the data

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 12,000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@revanth_kanteerava]

I want to extract removing the string : Your answer is correct I tried using below: (?

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 12,000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@mariavlogs327]

Is there a way to do this on a "Load More" button?

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 11,000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@myworshipresources]

I want to scrape for the scenario below: Scenario: A website has listed 10 articles' titles on each 1/50 pages. The full article can only be viewed when the article's title is clicked which would navigate to the full article page. Now, I want to perform this task. Step 1: Open the website page 1/50 Step 2: Scrape the article title 1/10 of page 1/50 Step 3: Click on the article title 1 to navigate to the full article page Step 4: Scrape the whole article from full article view page Step 5: Repeat Step 1 - 4 untill all data have been scraped from all 1/50 pages. Please solve this. This is very challenging.

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 5800+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@aydika1]

Had anyone else had a problem that not all of the elements got selected and later transfered into excel? I only have 1 page but only first 21 names got into excel

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 8000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@benfrankie6936]

how can we automate scraping from sub/inner pages of each profile and scrape phone numbers or emails of each one if available? thanks

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 4500+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@bandb1047]

please can u see easy way?

[@andersjensenorg]

Thanks for writing 😊 I'm getting more than 50 messages daily. While I read all of them, I can’t reply to everyone 😔 But I’ve created an RPA/Automation community where we’re 10,000+ RPA Developers helping each other with solutions and our careers. Here’s the video on how to join (the invitation link is in the video description): kzbin.info/www/bejne/roipq2CJbpuLfdE Kind regards, Anders

[@shaolin60]

Thank you for [0][0]

[@andersjensenorg]

A pleasure, Alex, happy to help 😊

[@ALVINMAN452]

Very very important video for we all. You should made a video only about REGEX it will be nice . Many thanks Anders Jensen . Obrigado. 👍👍👍👍👍👍👍👍👍👍👍

[@andersjensenorg]

Hey Alvin, thank you so much and RegEx for PAD is on my to-do 😊 Kind regards, Anders