Web Scraper pagination tutorial

[@aafibaloch654]

One of the best scraper i have ever used. This scraper saved my lot of time . I just use this and scraped above 2000 product details with 15 columns. This video deserve billions views. Lots of love , respect, support and best wishes for Web Scraper Team from Pakistan keep creating and keep helping us. More power to The Team.

[@taura101]

0:20 Method 01. Navigate pagination links while extracting product links 0:41 CREATE SITEMAP 1:06 add [1-20] for start url 1:50 Method 02 use link selector to navigate pagination pages 2nd sitemap 2:12 site url ONLY the 1st page 2:18 create link selector to extract pagination links 2:35 also make pagination selector child of itself and the start url so that the selector can find pages 9-18 in this example 3:00 now add the product selector that would extract product links 2:32 look at selector graph 3:45 run the scraper 4:02 3rd option scraping from page content that changes while page url does not reload on clicking pagination links because data dynamically loaded from the server

[@serkannn84]

This is great, super helpful. Also, your speaking is super understandable and explanations are super detailed. It is explained the same thing in 3 different ways. I could use the third solution because the website that I needed just let me go on that way. Thanks again and again.

[@beat5341]

This software is the dogs bollocks! This type of shit used to take me hours in Selenium with Excel VBA. Working out all the arrays and page elements. Incrediballs

[@rsstnnr76]

What about a site that doesn't have pagination, but data loads as you scroll down? For example, I want to scrape Google Photos to get a list of all of my album names. However, only a few albums load at first and then you have to keep scrolling down to see more and more albums. Is this possible?

[@user-nf9yc9pg1o]

what if the data is been loaded dynamically from the server but you have to scroll down and click on show more button and the old date you already scaped remains on the page

[@devtube6292]

What if the link of production wich i want To scrape its data open un new page, what can i do for that

[@whothinksforme]

How bout just a simple "Next" button?

[@dromus_]

very true

[@EmilNiemi]

Well when I use the method of doing it in the URL [1-300] example, it begins from page 300.. I'd like it start from page 1.

[@fahimimtiaz4532]

An excellent step by step tutorial. Thank you.

[@beyonddatascience8768]

Can you make a tutorial of how to use pagination with a Next button at the bottom? My scraper is getting stuck on page 5 when the results to go page 10. Thanks!

[@renderdreality]

The explanation needs more more. It does expplain what it does, but some examples you just cut off, but explain that you would have to do it for it to be complete. At 3:40 you don't finish the last step by adding the product detail page, and i'm not sure as a new user what it should be child ot. Does it have to be a child of root and all else as you have been doing or doe it only becomes a child of the direct parent? You need to finish the steps with no expectation that we will draw the correct conclusion, we are new users trying to learn a totally foreign idea (recursive due to pagination, etc).

[@rwky]

that's the reason why I'm in the comments... I can't figure out how to get data of products from pagination links..

[@DylanBarsby-vl6yq]

What if we only have left/right arrows instead of actual page numbers listed? I'm trying to find out how to deal with this.

[@wajidhassan789]

How to paginate with load more button or infitive scroll loading

[@domaincoasters8093]

how can i navigate through individual dates, I selected 1st Feb 2023 got the data then there are right and left arrow to navigate between individual dates, how can I click the right arrow for next date and loop it for xxx number of times. URL doesnt change.

[@jawadahmadkhan1076]

Third way, why were the product wrappers were selected, why not only the pages and product links ?

[@amarjagtap9447]

Awesome! You saved my huge manual work. Thanks :)

[@musanime1215]

how if the page is using ajax for pagination ?

[@frim3647]

This is incredibly useful, thank you so much!

[@AladinComadin]

page number range should be like this [1-10]

[@ruthmiller5740]

Thanks for this! I can't make it out in the video.

[@mejdiwassouf7991]

I want it to do pagination and also i want to visit every product and scrap things from the page inside , is that possible?

[@miguelrodriguez2349]

Then create a new selector inside that last element that is a link element

[@vincentamus]

@@miguelrodriguez2349 Please can you explain it again?

[@zackdattacker]

How is he auto selecting all the items on that page?

[@natesofamerica]

Tutorial videos should really be a bit slower and have highlighted points of what and where you're clicking. I have to constantly rewind to see what you did because you do it so fast.

[@Vinchek]

Hello. Who knows why the parser duplicates the products on the page?

[@Srinivas_Lakshman]

how to select multiple parent slectors, i am able to select only one at a time. i am on mac

[@tuya2023]

how long does it scrap for 28 pages normally?

[@catiformya1606]

Thanks you ❤

[@asgerlarsen2083]

Is it possible to scrap when a website uses scroll-down to load the next elements?

[@garp9433]

I wish I could add some random draw so the seconds aren't on the same millisecond tick because that's easy to catch if you make more than 2 calls they could easily detect a crawler...

[@darksideadv]

Depende el rastreador.

[@garp9433]

@@darksideadv sabes algún?

[@darksideadv]

@@garp9433 It happens that "good scrapers are paid", I have seen open source and they are good, but it handles a lot of code and other things. In short, there is no perfect scraper, but if you want you can try these: octoparse, scrapinghub, Data Miner and apify. I leave it to your choice, check well and do not start a test if you are not sure or you would lose those credits.

[@ShotaProductions]

i use method 2, the pagination can click automatically but it cannot go another link "a href", even though I return the whole table from the parent node, the child node doesnt work

[@CherryFox7]

When using element click, am getting only one page content... because pagination not like link... tell me the solution please

[@rexfarell]

This tool would be so much more popular if you had well designed tutorials...what a shame.

[@dawabhutia218]

why i dont have different types of element selection enabled on this tool?

[@barlospalos3780]

Nice work dude!!!

[@sandeepfrancis2253]

How to configure with next button? Also pls advise how to configure without creating duplicates...

[@zhangyi674]

Pretty good extension dude! good job!

[@sajawalmustafa4445]

Dear, i follow the 2nd step. Its very helpfull but its take long to scrape hundreds of data. How to scrap data in fast way please reply

[@adenmardomingo672]

i see there's only 4 pages in the pagination of the site. what if there are like hundreds and not all pages are shown at once?

[@AnsafP]

is it showing in link page 1 like the first method in the videol

[@stevenheintzman5956]

Hi, is the pagination feature only available in the premium plan? or is it also available for the free version?

[@latestgaming1774]

You type=Link function not working

[@rajataudichya895]

What if I have pagination inside pagination, I tried working the element attribute twice but it isn't working ?

[@darksideadv]

no funciona porque es free, si pagas te funciona. Es por los tiempos de respuesta.

[@smailkhan7647]

in parent selector I cant select multiple parent

[@darksideadv]

Para que funcione como lo muestran en el video debe ser de pago y tiene un máximo de 1.000 capturas.

[@Naiemaa]

some sites have a [Show More] button that downloads the next lot of item on the same page and keeps showing the same button at the bottom until all items are shown on the same page. how to handle that?

[@Naiemaa]

I have found the answer in the next video :P kzbin.info/www/bejne/lXTXk42rl5qhr5I&feature=emb_logo

[@tokoindependen7458]

how about if "page=0" is page 1, i try in shopee.co.id does'nt work

[@wilsonpool]

anyone knows what S P C means in the select button??

[@tad_tao]

Super. Parent. Child.

[@Axess-btlexpert]

Sadly very low quality video. Cannot see typing.

[@allabouttechurdu3675]

It barely scraps 3 pages if u have any solution than share :) :(

[@danlop_2023]

nice but google show me a captcha very quickly

[@crisxjanes7288]

How to set infinite scrolling?

[@pareokpareok9335]

how to handle dropdown pagination?

[@double2406]

When can I support folders?

[@9ebooks]

Hello sir nice work explaining i need help for scraping s all html embed video i can't select the selector lol

[@WebScraper]

Hi! Selecting elements within an won't work at the moment. That said we have made some progress regarding this. The latest dev version can extract data from an but you would have to write the CSS selector manually. Here is how to write it - forum.webscraper.io/t/web-scraper-chrome-extension-release-notes/86

[@patrickmccartney7544]

i know you might get it but you seem to just go over this so fast

[@zbabyface]

What about a "load more" button instead of pagination?

[@queseraseraozi]

In that case you better to try agenty.com I have the similar issue. I am looking for a good scraper to extract data from a dynamic website loading with ajax

[@raheebashraf3545]

@@queseraseraozi I can do it.

[@ThePutrified]

forum.webscraper.io/t/how-to-paginate-pages-on-a-site-that-the-url-does-not-change/144/2

[@user-ik2kt6dj2n]

if it was previous, 1, 2, 3, ...., 9, next.... it won't work!

[@thatWASSILLY]

I didnot work proper

[@KemoSays]

what if the urls don't change but i just have a button that says 'next'?

[@moebasij7443]

+1 Trying to do this on Google maps and this is the issue.

[@havoc127]

lying about the "ability " to scrape multiple [ages, with "pagination". don't think so, did so as per instructed, any links not written on web page is NOT i repeat NOT collected. So links betwen 3 and end, NOT collected from sveral un\dred. No parenting helps at all. .

[@sarabjitkaur-bw4vd]

help me

[@ogradus]

This is an API tutorial... not helpful

[@NetSkillNavigator]

damn.. worst video.. U could've zoom it