5 Things You Might Not Be Using in BeautifulSoup

[@ScottEllisInPismo]

Thanks, John. Your videos have been of great help to me. You're clear, concise and thorough. I really do appreciate all your content.

[@JohnWatsonRooney]

Thank you that’s very kind

[@ajithshenoy5566]

Hey John!! please create a tutorial on recursively scraping websites using scrapy , would be super thrilled if the content is dynamically populated

[@rrahll]

Liked and subscribed. Best channel about web scrapping in Python. Thank you!

[@barui7009]

Can you tell me alternate package of requests-html in python 2.7 version. I want to use python 2.7 but requests-html support from python 3.6 . Please give suggestions.

[@afrowave]

Thank you for this collection of tricks.

[@quytrannhat11]

Great tips, thanks you John !!!!

[@JohnWatsonRooney]

Glad you like them!

[@amineboutaghou4714]

Gréât tips and excellent video ! Many thanks John !

[@JohnWatsonRooney]

Glad you liked it!

[@lordmo3416]

live matches classes in sofascore returns an empty array from both beautifulsoup and scrapy. The past week has been a nightmare as i just started using python in december 2020. what am I overlooking? Thanks

[@JohnWatsonRooney]

That site is all scripts loading the data. I've had quick look and we can probably get the data from the API Endpoint - check out my video here: kzbin.info/www/bejne/iXa0m4F7ltyWm9E that explains the concept and see if that works for you

[@lordmo3416]

@@JohnWatsonRooney Words can't explain how much your help means to me. Thanks. I've been terribly irritable this past week, your response has certainly alleviated much of the grumpiness. I'll try it out this evening. Another thing. I struggled to install scrapy with pip install, but couldn't. I had to use anaconda. What's that about?

[@JohnWatsonRooney]

@@lordmo3416 no worries, i hope you feel better. Did you get an error trying to install scrapy via pip? unusal to have issues - can you pip install everything else OK?

[@lordmo3416]

@@JohnWatsonRooney yeah, i feel better, thanks. Im able to install everything up until now. I think I get an error: unable to build wheel for Twisted. then I get more gibberish.

[@tubelessHuma]

Very useful tips John. 👌 Thanks Dear 💖

[@wassuphomies263]

These are some awesome tips! Thankyou

[@JohnWatsonRooney]

Thanks!

[@ferilukmansyah3037]

that's a great tips, hello john, I want to ask if there is a recommendation for a library for parsing javascript, I use selenium, but after I discussed in the forum, the selenium library is not recommended for use on the server, are there any suggestions for libraries for rendering javascript, other than selenium, btw thanks for make this great tutorial

[@JohnWatsonRooney]

thanks - sure look into requests_html (puppeteer) or Splash

[@ferilukmansyah3037]

@@JohnWatsonRooney is this documentation..? pypi.org/project/pyppeteer/

[@ferilukmansyah3037]

@@JohnWatsonRooney is this documentation..? pypi.org/project/pyppeteer/

[@TheSalik1]

Which one is better for a professional: Beautiful soup' Selenium Scrapy

[@JohnWatsonRooney]

Scrapy definitely!

[@TheSalik1]

@@JohnWatsonRooney thanks for your valueable reply

[@technicallymind201]

That's great video for us to learn about bs4 but sir i want to make on api for amazon login but i can't can u please help me

[@absoluteRandom69]

Hello John, I'm not able to crawl a website because of captcha. Who should I handle it?

[@JohnWatsonRooney]

There are captcha solving services that you can use - they work in a few different ways but will also cost a little

[@absoluteRandom69]

@@JohnWatsonRooney Could you please try to make a video on that. Thanks

[@aogunnaike]

Great stuff 👍

[@mushinart]

Good stuff,thank you bro

[@sudhanshuyaa]

Hi Jhon Thankyou such type of videos It's really helpful for us I need your help budy I wanna scratch Data from Taobao And Lazada type website with multiple pages but not able to do that. I need from these sites like Tital,Price,Seller Name,Seller Url and many more things please help budy Please please help brother Thankyou in advance

[@sudhanshuyaa]

Please help jhon it's about my job Thankyou

[@picassoofai4061]

Can you give us your github?