Is Your Scraper Slow? Try THIS Simple Method

[@hicham_6544-_-]

I have been following you for about 3 years, thanks for all the information, we wish a video on how to spend your day 😊

[@muhammedjaved4322]

You really deserve 10000000000 subscriber

[@Mr.AIFella]

Q: I'm a beginner in the web scraping world as I entered it last February and I was trying to scrap a text from a single infinite scrolling page that needs two operations: one is clicking show more hyperlink (to show more hidden text for the reader). Two is scrolling the page down to show more content. Do you have a video reference that you suggest me to watch for this purpose. Appreciate it in advance, Best

[@Extrey]

Amazing!!! I have sync playwright parser, and it's little bit messy and big, so rewriting it to async will be week of time at least, and tthreading will be powerfull solution. Thanks for your videos, insainly usefull, as ussual!

[@irakli1264]

Thank you John. I am considering scraping as a career path, but unsure about it. Would be nice to hear your opinions about web scraping career.

[@exqvision]

This is great. Do you have a link to the code for review?

[@mintydevdaz]

have you considered using the niquests library in place of requests?

[@JohnWatsonRooney]

No I haven’t heard of it - looking at it now will definitely give it a go thanks!

[@razapoetra9355]

I always get a new library every time I am going to comment section. 😂

[@itzcallmepro4963]

also what's the difference between using async_playwright api with asyncio and using sync_api with threading , when i used sync_api with threading , it really did open 4 browsers , but only 1 was scraping while others did nothing until the first one is finished .

[@itzcallmepro4963]

i still don't undetsand the difference between it and using asyncio with httpx for example , both almost works the same way i think , it waits for a thread to sleep and then runs another one , also in async when you wait for a request another process runs ,

[@MahmudNuman]

loved it 🥰

[@patrickavis5475]

I'm being slow...can someone explain the use of the proxy service/server here?

[@berkay.digital]

Each request is sent through a different IP address to make it appear as if it's coming from different users. This prevents the website from blocking you.

[@bakasenpaidesu]

;)

[@alexdin1565]

please can you make a video on how we can calculate how much 1k requests can costs I go to the nodemaven website and they say 5 GB for $35 I'm planing to sell scraping service but if 1k costs $35 its very expensive

[@berkay.digital]

When you use proxies, the amount of data transferred varies depending on whether you're rendering a website or requesting data from an API. If you're rendering a website, it generally takes around 2MB per full load. However, if you're requesting data from an API, the amount of data transferred is much less. It's hard to determine the cost of the job as it depends on the specific needs of the task.

[@pypypy4228]

My third like 😊

[@JohnWatsonRooney]

Thank you!