Async Requests Made Simple - Grequests for Web Scraping with Python

[@davichiqueti]

Thanks man! I am a Junior Python Developer and this have helped so mush in just 8 minutes!

[@junaidmughal3806]

such an underrated channel, keep doing it mate, hopefully your channel will blow up

[@JohnWatsonRooney]

Thanks!

[@pythonantole9892]

Amazing. I keep learning new things on this channel. Had never heard of grequests until today.

[@jp-pn6wk]

what is the difference between grequests and asynio in terms of performance

[@bharathg8072]

Very underrated channel. Amazing content.

[@mattbass4807]

ahh I was just wondering about this Thank you so much you have helped me be way ahead of the curve in my data analysis class in college!

[@celerystalk390]

You keep impressing us with your knowledge of web scraping. Keep up the great work!

[@trustedapp6532]

Amazing connect and nice explanation with correct example Keep doing great things john...

[@களவையும்கற்றுமற]

I was looking for this. Thank you very much for sharing your knowledge.

[@amineboutaghou4714]

Great video John ! Many thanks for the tips !

[@ferilukmansyah3037]

keep learning from this channel

[@mattmovesmountains1443]

Another banger. This'll be a nice upgrade for my data science scrapers. Just gathered 20 pages worth of house listings, 50 listings per page, then went into each of those links to get the data. Prob took 15 minutes. Curious to see how this cuts that down!

[@beastvirus]

Waiting for you to reach 1M!

[@bolatbekminbayev9217]

Finally, thank you so much!

[@ugurdev]

This blew my mind, much faster than even concurrent, surely a semi decent website would block you right away using this though. Combining this somehow with rotating proxies would be amazing though.

[@JohnWatsonRooney]

Yes - that’s exactly what you do!

[@ugurdev]

@@JohnWatsonRooney that would be a nice tutorial to have.

[@JohnWatsonRooney]

Working on something - a few weeks away I think!

[@alizimmx4776]

Thank you sir definitely learning lot of thing plus will use in my work !

[@fraddygil]

amazing.Thanks for this video!

[@kevin-wg5iv]

Thank you, always amazing content

[@pandalanhukuk804]

I think you should do a course and teach this really interesting insides. Could be on Udemy or on your own platform if you don't want to get cut from Udemy provisions. Thank you for the content.

[@JohnWatsonRooney]

Thanks for watching! I’m planning a course but it’s still very much in the early stages

[@pandalanhukuk804]

@@JohnWatsonRooney im staying tuned. Grab some data, you will be surprised. :-) how much value you can make. :-)

[@tnssajivasudevan1601]

Great Thank u for sharing the knowledge.

[@mukeshwarsingh270]

Bro, I want to hug you so hard you can't breathe... THANK YOU :)))))))))))))

[@tombrady2442]

Hey, John - awesome stuff as always. I was wondering: is async and all the async scraping libs (aiohttp, grequests) aren't the same as scrapy? I mean aren't these all running code concurrently?

[@sayidinaahmadalqososyi9770]

thank you sir for you sharing, can i ask about thread, when i make requests,a lot requests ,and if i set thread much more the proccess more slow, i hope u understand my english so bad

[@KhalilYasser]

Amazing. Thank you very much.

[@ОлегАбрамов-к5д]

Thanks for the cool tutorial! Can you please tell me how to pass cookies into a session, so that each request contains them?

[@nsnilesh604]

Awesome sir 👌

[@ammaralzhrani6329]

Thanks , a im trying to understand async, I would ask about this library how to add headers to it ?

[@CodePhiles]

great approach, thanks for the video, but it may be with limitation with some webistes..right

[@shaharrefaelshoshany9442]

AMAZING, great content,

[@gisleberge4363]

Using this grequest and scrape asyncronicly, would you say there is a chance of putting pressure to the websites that we're scraping, and more easily gjetting noticed and possibly blocked from the site? Like you say, everything goes faster...

[@JohnWatsonRooney]

yes absolutely, ideally you want to use this with some rotating proxies

[@xaviruiz8345]

Amazing!!! I'm learning so much with your videos.. I really appreciate! Have you ever tried to scrape the API of IKEA (stock availability per store)? It can be a great challenge:)

[@abdullahsiddique7787]

I need to call 13 k urls , and then process the response in pandas and write them to a csv . Do u recommend the async method or a multi thread approach ? Pls do let me know . Pls note the total response data size may go upto 70 80 gb , so not even sure if the response list variable can even hold it .

[@nightowledits5708]

So how to this output in a webpage... I'm currently doing my final yr project based on product comparison I done the scrapping file but it's output is showing in command prompt I want to show it ina webpage... Could u do a tutorial on that... That would be very helpful

[@marsin23]

funny how u did list comprehension for the requests and not for the get_urls with single line return ^_^ nonetheless great! haven't heard about grequests

[@JohnWatsonRooney]

haha oh yeah, well spotted

[@tarekrahman1]

is there any way to send async html requests like requests_html for javascript rendering?

[@marvalmoments2099]

can we use proxy in grequests moudle?

[@RamMohan-xu3ey]

Thank you so much sir..

[@arunavadatta7734]

Hello John...I need some help regarding async ...how can I share my program to you?

[@MessWithAI]

sir can u say how to use asyncio with proxies its very tough and facing lots of issues because of this

[@ferilukmansyah3037]

interested, thanks

[@DM-py7pj]

for parsing could you gain anything by using multiprocessing?

[@JohnWatsonRooney]

Yes you probably could, but I'm not sure it would be worth it until you get to a massive scale as the time saved would be minimal

[@DM-py7pj]

@@JohnWatsonRooney thanks for the great videos

[@rangabharath4253]

awesome

[@venolimold2179]

Can you make video for videos web scraping?

[@heshamabdellatif196]

Great