How to scrape website links with Power Query

[@GabeTek]

I saw the post from Chris Webb but I didn't get it right away. You just made it so clear! Thank you!

[@CurbalEN]

Fab!!

[@richweinberg6355]

This worked great. Thanks. At first this didn't work for me, but it was because the links I wanted were only relative links, meaning they didn't contain the website address or . I modified the text from: {"Link", "a[href^=""http""]", each [Attributes][href]} to {"Link", "a[href]", each [Attributes][href]}

[@peterkrkos6489]

Many thanks, this helped me

[@Chriiichriii]

Same issue, thanks you bro !

[@EricaDyson]

Well a big thank you to both of you! Never would have discovered this without you!

[@CurbalEN]

I was lucky I saw his code :) /Ruth

[@paspuggie48]

I'm sure I commented but I can't see it. Anyway this is brilliant Ruth and had read Chris Webbs post before but your videos makes it easier. I managed to extract 140 URLs from one webpage...Cool!

[@CurbalEN]

It definitely is! 🥳🥳

[@paspuggie48]

@@CurbalEN I had to use Rich Weinberg's edit version but worked a charm. Does not work on some webpages I've also tried though. Still, it has helped wonderfully Ruth :)

[@CurbalEN]

Oh, maybe chris webbs trick doesnt work on all sites. Glad there is a fix out there :)

[@paspuggie48]

@@CurbalEN Well, ironically the webpage I tried was Power Query's own Function Reference page, even with Rich's solution I couldn't get it to extract the url's LOL

[@CurbalEN]

😂😂 Check the html code, it is there somewhere, or see if it follows a structure you can replicate manually.

[@diegolozano2397]

OMG, Thanks,, Thanks.....sooooooo usefull video

[@nikakalichava8012]

very useful, thanks,

[@ayobamiologun9184]

Cool trick... Thank you Ruth.

[@CurbalEN]

:) /Ruth

[@ezequielellena]

Hello, what KPI or analysis could be done with that table. I have the same case and I don't know what to analyze.

[@chamelion100]

You are a great help mam............thanks a ton

[@CurbalEN]

My pleasure , happy it is useful :) /Ruth

[@AnalyzeIt_Nael]

This is an awesome video. Thank Ruth

[@CurbalEN]

Thanks! /Ruth

[@miguelcalejo1770]

I'm trying to scrape data from a web page which displays the 10 top items and there's a button "View more" to see more items (till 100) which I open, but powerquery only scrap the top 10 and displays "View More" as a line. Is there a solution for this? Thanks

[@bpdeepakbp]

Thank you for this video. this is really good one

[@CurbalEN]

Yey! Happy it was helpful:) /Ruth

[@oshiowueme254]

hi, if the column1 could not extract the exact text, how could I do, if I split the column1 after fisrt step?

[@MoviFrenzy]

Hi Ya plz help me the web page scraping which has credential page

[@usmanmirza8966]

I am trying to do the same but the navigator doesn't load the entire page as there is a script arrow at the bottom of the page hence showing only first 40 rows. Please advise.

[@iyasabdulrahman]

Breliant, thanks Ruth, 👍👍👍

[@CurbalEN]

🎉🎉🎉 /Ruth

[@neerajahirwar8600]

Can i get the link of all PDF using same steps

[@alreadydeadfunk]

When I saw the title of the video (before watching it), I thought about Chris' blog, too.

[@CurbalEN]

He is the best :) /Ruth

[@TheVamos777]

Same here

[@Softwaretrain]

You are amazing...

[@CurbalEN]

:) /Ruth

[@jerrykappa6899]

Awesome 😎

[@CurbalEN]

He is :) /Ruth

[@TheWandererRises]

Excelente!! Graciaaas!!

[@CurbalEN]

💃💃💃

[@walidkhlil2872]

thanks again and again

[@CurbalEN]

:) /Ruth

[@CeliaAlvesSolveExcel]

Beautiful!!

[@CurbalEN]

Simplicity is always best :) /Ruth

[@cokemaya]

Awesome tips as usual! I was thinking about going a level deeper in the hierarchy and scrapping those URL's, for instance, creating a table from a webside "www.example.com/Address/number" and then create a function that calls the column from the previous table which contains the hyperlinks with different "Addresses" & "numbers". Obviously they have to share the same structure to allow me to put out the same data. " www.example.com/"&Column_with_URL's&" Well I do not known if that make sense, but it would be great to learn that. Congrats for you channel!

[@CurbalEN]

Yes, like this? m.kzbin.info/www/bejne/bIG9hI1raM6GpdU /Ruth

[@wimvanderlinden3080]

Where can i load these pbix files? Very nice film!!

[@CurbalEN]

If you go to curbal.com > Resources > download center and then Community downloads, you can get the filés from there. /Ruth

[@wimvanderlinden3080]

@@CurbalEN no 140?

[@CurbalEN]

Exactly!

[@sheilahart5989]

Very cool

[@CurbalEN]

❤️ /Ruth

[@CPatwal]

Thanks Ruth, will the website owner know that his/her/their website was scraped ? I mean I would want to just avoid getting into trouble :-)

[@CurbalEN]

It is my website :) /Ruth

[@bartdepauw1156]

No, that can't be seen, as the links are obtained from the (already downloaded) page. If you access each individual link, that can be seen in the web server log.

[@CurbalEN]

Better answer, thanks!

[@kaustubhk8648]

subtitles destroying video

[@elchin9748]

Nothing special. The most important is how to get data from each scraped link (and you can do it with custom function)