Python: web scraping, pagination & headless with Selenium

[@Arron0075]

this was really helpful, simple and on point, thank you so much.

[@juliorosales1546]

This was great and insightful. Can you do what to do if there are no numbers in the next to page? Meaning all you see is the next button?

[@codingisamazing]

what you can do is, use a while loop and side it use a try a except block - inside the try block, add the click on the button code - inside the except block just add break - the extraction code add it inside the while loop and after the try and except block when it reaches the last page where there is no button it will go to the except block, it will break. While True: Try: # here the code to click on that button except: break # here the code to extract data from the page I hope you understood