I appreciate that you keep things simple and colloquial. Many teachers try to sound smart and they just end up confusing their audience. You are a good teacher.
@jeremyjamero1993 жыл бұрын
Awesome! You make my small understanding of Python seem like I've been coding for years!
@subhamsubhasispatra3 жыл бұрын
This is really an awesome tutorial . Please make a playlist on mechanical soup . ❤️❤️
@PythonSimplified3 жыл бұрын
Thank you so much Subham! May I suggest you another very cool web scraping library, even more powerful than Mechanical Soup?? Have you tried using Selenium? It scrapes information out of dynamic language websites (Mechanical Soup can only do HTML & XML) and you can log-in and out of any web account - in short, it's the most powerful scraping library I've ever tried! If you want to learn how to learn how to web scrape Instagram images, checkout my tutorial on Medium: medium.com/analytics-vidhya/web-scraping-instagram-with-selenium-b6b1f27b885 Or checkout the video version (I still recommend the article though, it's more accurate and less controversial 😄): kzbin.info/www/bejne/n3uqp4x4fZ6gmbs
@namastemukesh4 жыл бұрын
Please keep posting projects more frequently. That makes a lot of sense for us. More realistic and more usables.
@PythonSimplified4 жыл бұрын
Hi Mukesh, I'm gonna try to do my best to post videos on a weekly basis. The feedback I'm getting is incredible, I didn't expect it when I started this channel and thank you very much for commenting and sharing your thoughts! :) With that said, I'm currently trying to figure out how to film in 4k as I got a new camera. It might take me a few extra days, but I think it's worth it for the long run. Once I have it figured out, I'll advertise an organized upload schedule so that everybody knows what/when to expect new videos ;)
@namastemukesh4 жыл бұрын
@@PythonSimplified You have the skills to make people understand easily. I do have ideas about small projects, if you want to discuss.
@chiranjeebroychowdhury77593 жыл бұрын
Easy Peasy, Lemon Squeezy! Nice breezy video yet again! I have used Scrapy and Beautiful Soup a lot. Never worked with Mechanical Soup. This was easy. Also I usually download and save files using urllib. Thanks again Mariya!
@webscrapingwithandy21104 жыл бұрын
⭐️ Timestamps ⭐️ 3:00 - Search for a term on google 3:45 - Extract source HTML from the web page 6:29 - Navigate to the new page 7:08 - Look for all the image elements on the new page 9:01 - Make a local directory for the images 10:19 - Download and save all the images
@PythonSimplified4 жыл бұрын
Sorry Andy, somehow I missed your post, thank you so much for the timestamps - you rock!! ❤
@CdaPlays4 жыл бұрын
Hi, New Subscriber here..from Facebook python group
@PythonSimplified4 жыл бұрын
Thank you so much for subscribing! Welcome to the channel! :)
@CODTALES-KILLSTREAKS3 жыл бұрын
I love this channel! I’ve watched many many python videos and I am in love with this channel
@ankanmazumdar50002 жыл бұрын
Thank you dear...you're most beautiful data acience creator in this universe ❤️
@jamalhasanzakarneh98373 жыл бұрын
Thank you very much. It is very nice and helpful one. I used the same code but I got 20 images however you got 3057 images; could you please explain why?
@siamahmed82873 жыл бұрын
You explain things really simply. That's why i like you. But would you mind if i give you a suggestion that will you change your wallpaper? Something fancy? 😇
@PythonSimplified3 жыл бұрын
Hi Siam, this video was filmed almost a year ago! 😁 Not only do I have a new wallpaper now, but I actually have a new computer! (and much much better videos about webscraping by the way 😉) Checkout both of my Selenium bots: Instagram Scraper: kzbin.info/www/bejne/n3uqp4x4fZ6gmbs Facebook Scraper: kzbin.info/www/bejne/iaS7lKylmtuejJI An additional Linkedin bot will be released on my channel tomorrow 😉
@siamahmed82873 жыл бұрын
Hey I know. I've seen your videos. I don’t miss your videos
@Scal3 жыл бұрын
Hello ! EDIT : I did a new scraper thanks to your tutorial on Selenium and it worked !!! Thanks a lot !* Sorry I am doing same kind of thing on a page with a form but I got a problem. I think my problem is the same as "how can I go on page 2 from cat search results 1st page?" I would like to fill in a form on a 1st, submit it, and move the browser to a 2nd new validation page. the 2nd page is just summarizing what I inputed on the 1st page, and asking me to validate with a checkbox and a submit button. I get the new 2nd validation page url correctly with browser.get_url(), but when I do browser.launch_browser() I m still on the 1st form page response = browser.submit_selected() print(browser.get_url()) new_url = browser.get_url() browser.open(new_url) print(browser.get_url()) page = browser.get_current_page() print(page) browser.launch_browser() both print(browser.get_url()) are showing the 2nd page url, but browser.launch_browser() still display the 1st page. the print(page) still show 1st page html and when i try to select the form with the checkbox on the 2nd page it can't find it. How can i make the browser go to the 2nd page and be able to check the checkbox on the 2nd page? I think my problem is the same as "how can I go on page 2 from cat search results 1st page?"
@sagarsriva3 жыл бұрын
watching your video for the first time. you should make more such videos! great learning
@thebuttnakedpodcast30063 жыл бұрын
You make everything feel so lite and easy, thank you. #best teacher ever
@PythonSimplified3 жыл бұрын
Thank you Harold! 😃 Keeping it simplified!
@richardmodad71664 жыл бұрын
this was a really good scraping tutorial
@PythonSimplified4 жыл бұрын
Thank you Richard, I'm glad you liked it! :)
@Orion3000k3 жыл бұрын
🤯🤯🤯 completely awesome. Great explanations. This is what we devs dreamed of. Thanks tons.
@essetee3 жыл бұрын
Nice tutorial, but i don't get the url printed, but instead he starts my browser and show the url's. Also better to change code for creating directory in : #create the directory if not os.path.exists(path): os.mkdir(path) to avoid errors if the dir exists.
@faceprofesor2 жыл бұрын
Very neat. Thanks for the tutorial. I have a question: how many images are being downloaded over the scrapping process? Can one set the number of images downloaded?
@JohnWatsonRooney4 жыл бұрын
I’ve not heard of mechanicalsoup before look interesting I’ll give it a look thanks
@baconsledge3 жыл бұрын
Your videos are excellent. How about recommendations for packaging stand alone Python executables?
@TrevorHaagsma3 жыл бұрын
Auto-py-to-exe is awesome
@digigoliath4 жыл бұрын
From now on. I will be drinking Mechanical Soup!!
@PythonSimplified4 жыл бұрын
hahaha good luck surviving this! :D
@digigoliath4 жыл бұрын
@@PythonSimplified LOL
@mrsuncure12593 жыл бұрын
TY! for teaching me this. And also thank you Rohit & Benny for the hint.
@Grinwa Жыл бұрын
What if the image is not stored in the img tag on the website...? I faced this problem this morning the The website i wanted to scrap store's image url this way that kind of websites uses css And using sop the regular way can't scrap images from websites that uses this method I have managed to find a way to bypass that, am planning to release the code on GitHub I spent the whole day working on it and learned a lot to be honest
@shuvrodas65763 жыл бұрын
Your videos are great. You can teach so much in a few time that amazed me. But one thing bothers me that I cannot see the screen clearly as the font size is too short to understand. I only understand it by hearing from you. Would you mind increasing the font size? Although, This was a complete tutorial. Thank you for the video.
@overcomermindset3 жыл бұрын
Woman. You are a beast !!
@TSerumula2 жыл бұрын
Wow . Thank you. You are really good at teaching.
@tradewithdani1224 жыл бұрын
how to give different headers (user-agent)with mechanical soup to the website?
@PythonSimplified4 жыл бұрын
Hi Danyall, this should do the trick: import mechanicalsoup browser = mechanicalsoup.StatefulBrowser() browser.open("my_website.com/") browser.set_user_agent('my_agent') Let me know if it worked! :)
@pavanghai4 жыл бұрын
Simple N sweet, thanks for sharing specially using Jupiter , I prefer using google Colab
@PythonSimplified4 жыл бұрын
Thank you Pavneet, I just saw your comment on FB, I'm glad to see you here tool! I'm usually a huge fan of Colab as well! The free GPU option is incredible! however, because this video involves a local directory I thought it will be much easier to do with a local interface... I guess mounting Google drive is not really that difficult, but Jupyter is a bit more comfortable on that aspect :)
@akaabdullah3 жыл бұрын
Please make a video on how yo bypass google recaptcha with selenium or any other automation tool with python that will be great thank you in advance
@PythonSimplified3 жыл бұрын
Breaking a Captcha is no job for Selenium 🙃 It much more complicated than accessing the DOM! I'm actually about to test a really cool tool (ScraperAPI if you heard), it should be able to break captchas in no time! 😁 I'll definitely film something about captchas soon, stay tuned! 😉
@tothandras81603 жыл бұрын
thank you for your demo. (what I would not do is to re-assing the value of the same variable with different meaning though; it is a bit difficult to read the code later for me.)
@rahulkmail2 жыл бұрын
Nice tutorial
@aHardReset4 жыл бұрын
Cool, I suggest add page scrolling and bigger versions of images :)
@PythonSimplified4 жыл бұрын
Thank you for the feedback Aaron! :) The images are small for a purpose, when you feed them into a dataloader for training it is common to resize/crop them into a size of 28x28 pixels (like the famous MNIST datasets for Machine Learning). But if you're not using these images to train models - it's definitely best to get their bigger version. It just involves a bit more steps, because each of the images is located on a different website, so it makes it more difficult to automate the process. If you end up coding this, please share it with me, I would love to see what you came up with! :)
@devakinandanrakoti18913 жыл бұрын
It will be better watching if you can zoom in more. Some commands are not clear while watching.
@PythonSimplified3 жыл бұрын
Hi Devakinandan, have you seen any of my newer videos? I zoom in quite a lot there, the fonts are huge! I've filmed this video almost a year ago, so many things have changed since 😉 Checkout my newest web scraping tutorial with Selenium, I think you'll find it very interesting: kzbin.info/www/bejne/jpWphqN8iqmnbpo
@kostas69152 жыл бұрын
Hello a technical question: can we web scrap an html table when a login is required for access to that page? (assuming we have the credentials ofcourse)
@user-mf8gg8dj4x2 жыл бұрын
ты моя умница
@kennethstephani692 Жыл бұрын
Great video!
@fazlaynur45093 жыл бұрын
love your tutorials
@PythonSimplified3 жыл бұрын
Thank you! 😁
@iansjackson3 жыл бұрын
HELP.... using VS Code. After browser initially launches it sticks on the search page with 'cat' sitting there in the dialog box. If I press enter then I get my cat images, but automation halted at that point. Any ideas?
@mehdismaeili37432 жыл бұрын
Hi thanks for this great video.
@digigoliath4 жыл бұрын
🤩 Nice. Soup is Good!
@alixaprodev3 жыл бұрын
Marya keep it up
@PythonSimplified3 жыл бұрын
will do! :)
@madslug71394 жыл бұрын
YOU ARE A DYNAMITE 😎🔥🧨🙏🤘!!!
@PythonSimplified4 жыл бұрын
Thank you very much Mr. Mad Slug! :)
@mockydigital2 жыл бұрын
hello... I love your tutorials. Quick question though how can we make the images scrapped HD.
@faisalee Жыл бұрын
Awesome!!!
@susmitbhamare4 жыл бұрын
Waiting for more frequent videos.
@PythonSimplified4 жыл бұрын
Thank you for your feedback Susmit, I will try to do my best to post videos on a weekly basis! :) I'm currently figuring out my new 4k camera and how to provide better quality videos for you guys, it might take me a few extra days, but in the end I believe it will really improve your viewing experiance. Once I figure it out, I promise to advertise an organized weekly upload schedule and push the channel forward :D
@susmitbhamare4 жыл бұрын
@@PythonSimplified Awesome 😍. Eagerly waiting for the content🥺🥺🥺
@mytoptechs2 жыл бұрын
Google must have updated their code as the URL won't return anything back. I just get output of the url that was used and not the page contents.
@basheersoft7118 Жыл бұрын
thank you
@mapachu69594 жыл бұрын
nice!!! I love your tutorials
@haydnrayturner138311 ай бұрын
Can we set he minimum size of the image to download?? And can we increase the number of images we download?? Complete noob to coding so any help would be amazing!! Thank you!!
@rverm10003 жыл бұрын
great video. but wasnt able to resolve the error. attrib error partially initialized module. mechanicalsoup has no attrib statefulbrowser( likely due to a circular import)
@albertgutierrez8862 жыл бұрын
Nice tutorials 👏👏
@alixaprodev3 жыл бұрын
Thank you so much ❤️. It really helps me
@PythonSimplified3 жыл бұрын
You're welcome, I'm really glad you found it helpful!! 😀😀😀
@TrevorHaagsma3 жыл бұрын
I wasn't able to use Mechanical soup for NTML AUTHENTICATION, any ideas how this could be done?
@gero80493 жыл бұрын
Oh my god. Why didnt i find this before. Instead of Indian guys I much prefer this beautiful ladie explain me things.
@mushinart3 жыл бұрын
Try requests-html .... This one is truly nice
@PythonSimplified3 жыл бұрын
Thank you for suggesting Nasser! I'll take a look shortly (I'm familiar with the get requests, but requests-html is brand new to me), Thanks again! 😃
@mushinart3 жыл бұрын
@@PythonSimplified sure thing. You will like it , its amazing. I teach python and specialized in coding training and cyber security. Please check my website : www.dotslashhack.io Love your channel and content. Truly amazing and i really appreciate the effort. All the best. And if there is something I can help you with regarding creating a good content in python, I'll be glad to help. Also i co own an arabic blog about python: www.pythonat.com Please check us out. Love and respect N
@darrickhatten70463 жыл бұрын
Ty
@funtooz95133 жыл бұрын
Hi need help? can you please tell me how can i find or extract [ href target="_blank"] links ??
@PythonSimplified3 жыл бұрын
Hi Yuvraj, you can use some of the syntax from this video to target the blank anchors, try using the following commands: page = browser.get_current_page() all_anchors = page.find_all("a") And once you collected all the links on the page, you can only include those who have a target="_blank" with list comprehensions. blank_anchors = [link for link in all_anchors if link.get("target") == "_blank"] I hope this helps, let me know if it worked! 😁
@funtooz95133 жыл бұрын
@@PythonSimplified Hey Thanks a lot, it works.. Thanks again.... !!!!!!!!
@justqtip2 жыл бұрын
very cool
@user-hd3pz2ow1b26 күн бұрын
thank u
@Planet_Xplorer3 жыл бұрын
Good work
@janduplessis13573 жыл бұрын
Love the tutorial, love all your videos btw You have a great way of explaining Python. What do can we do to download more images and not just the first 20 or 30. I know Google loads them dynamically - is there a feature in MechanicalSoup that allows for this. Thank you
@janduplessis13573 жыл бұрын
Found the answer in your reply to another user: Selenium!!
@basheersoft7118 Жыл бұрын
رائع
@ebq82813 жыл бұрын
Amazing ❤️😘😘😘
@DonEdward Жыл бұрын
"Dabble-yew, get" 🤣😂😂
@manitthatrimontrichai84464 жыл бұрын
Cool !
@gabrielfernandez96603 жыл бұрын
Ella tan hermosa la amo !!
@abhishekjoshi40913 жыл бұрын
Can I do this in gitpod too or online jupyter notebook ?
@dejohnny23 жыл бұрын
@Python Simplified that is a big ass laptop! 😁
@rangabharath42534 жыл бұрын
Awesome
@PythonSimplified4 жыл бұрын
Thank you! :)
@bedramtamang14394 жыл бұрын
Awesome video
@PythonSimplified4 жыл бұрын
Thank you! :)
@bedramtamang14394 жыл бұрын
@@PythonSimplified If you got time I request you to make video on infine scrolling, logged in system, dynamic content like javascript content etc.
@BeAPythonDev4 жыл бұрын
Subbed from FB. I'm going to need to make sure I'm not releasing the same vids as you ;)
@PythonSimplified4 жыл бұрын
Even if you do it's ok, the same concepts can be interpolated differently by different people, some will find your explanations easier and some will find mine. The more information we release put there in the web - the better :)
@desarrollojava2 жыл бұрын
👏👏👏👏👏👏👏👏👏
@invinciblenoman3 жыл бұрын
why does it save only 20 images only?
@PythonSimplified3 жыл бұрын
To include more images, you can scroll down with: browser.execute_script("window.scrollTo(0, document.body.scrollHeight);") If you'd like to scroll down multiple times, try looping over a range: for j in range(0,10): driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") time.sleep(5) //wait 5 seconds that way you can include as many images as you'd like, depending on the number of iterations. I have a video premiering on it in a bit, checkout "scroll to the bottom of the page" in the timestamps 😀 kzbin.info/www/bejne/iaS7lKylmtuejJI
@invinciblenoman3 жыл бұрын
@@PythonSimplified this is so great ❤️❤️❤️
@PythonSimplified3 жыл бұрын
@@invinciblenoman no worries! 😁 And sorry for the long wait, it was an older video of mine so I don't check it very often for new comments 😅
@invinciblenoman3 жыл бұрын
@@PythonSimplified yes I know I was searching your email id to send you this question in writing 😝
@PythonSimplified3 жыл бұрын
@@invinciblenoman hahaha I'm actually getting bombarded with questions on my public email from Github, so your best bet to get a reply is here on KZbin (on recently uploaded videos, I guess)😉 I'm also chatting live each time I run a Premiere, so I'm available for questions then. And soon I'll start streaming live sessions, so that all questions would be answered right away (just waiting for my new monitor and keyboard to arrive before I start doing that 🤩🤩🤩)
@shuvrodas65763 жыл бұрын
You should zoom your screen a little bit.
@hinomuratomisaburo49012 жыл бұрын
Cool 👈 content, cute teen 👍😎
@omkardalvi15883 жыл бұрын
Mam please make a gui chatting app using socket and tkinter please
@umer.on.youtube2 жыл бұрын
You were still on Win8 in 2020 ??? 🤨🤨
@RedStefan3 жыл бұрын
Next video is on Chicken Soup, right?
@muhammadzaakirmungrue31463 жыл бұрын
nice outfit 😉
@rohitkumarrajak25294 жыл бұрын
Will you try to do one more this. Using machinicalsoup try to login into Instagram. If you read this message then like this.
@PythonSimplified4 жыл бұрын
Hi Rohit, absolutely! I'm going to create a video for webscraping of dynamic language websites with Selenium. I wish Mechanical Soup could help with Instagram, but it is not supporting REACT, only HTML and XML unfortunately :( Selenium is the way to go if you want to try it on your own in the meanwhile... filming this video might take me a few weeks, I'll give you a shout once it's ready :)
@rohitkumarrajak25294 жыл бұрын
@@PythonSimplified I tried many times with Selenium but it's not working at all. Some errors always comes. I got a pip library which is working perfectly that's "Instagram-scraper" in github that working perfectly without any loging process if you want to make a video on that try to use "Instagram-scraper" code. I hope you are going to make video as soon as possible. Thanks
@PythonSimplified4 жыл бұрын
@@rohitkumarrajak2529 Let me look into it, I'll send you a solution before I film the video, I don't want you to get stuck waiting for me to design pretty graphics and write a fancy script - I could probably send you an implementation in code before the video i even in the making ;) Give me a few days to figure it out, I've seen people scraping instagram so it must be do-able!
@rohitkumarrajak25294 жыл бұрын
@@PythonSimplified i have one more question. if you want help me then i will ask. and what about insta scraping
@PythonSimplified4 жыл бұрын
Hey @@rohitkumarrajak2529 you'll be able to login to Instagram with the following code: colab.research.google.com/drive/1S_MxiCixGACGGAxkI7P114akwoZwMesv?usp=sharing If you implement the same methods of obtaining the React code, you can figure out the rest... if not, just wait for the video ;)
@mattmovesmountains14433 жыл бұрын
When you had searchword=cat, and then added the 's' and said "I'm going to concatenate" I started laughing at con 🐈 enate
@PythonSimplified3 жыл бұрын
Hahahahahahaha that's awesome!! I didn't notice it 🤣🤣🤣
@lovedale283 жыл бұрын
Why don’t you use a web scraping tool instead of writing all those code😊
@feliuswyvern71893 жыл бұрын
Well 1, because people like to code and do things themselves. It's also more rewarding when you code it yourself. 2, because you gain more experience and knowledge of how it actually works, and learn how to code this app yourself. And add in your own edits/tweaks to the program vs downloading what someone else has already made
@jeanmi81843 жыл бұрын
Oh une canadienne
@slayerification13 жыл бұрын
I wish one day I can have a girlfriend like you!
@jpongthep4 жыл бұрын
thank you
@PythonSimplified4 жыл бұрын
Youre welcome, I hope you found it helpful! :)
@hiseggcelency4 жыл бұрын
Hi! I have a problem, so after i do browser.get_current_form().print_summary() nothing happens, when i try to print it it says "none" i'm using visual studio code, any idea?
@PythonSimplified4 жыл бұрын
I can see how something like this would happen if you didn't include the following lines prior to this command: browser.get_current_page() browser.select_form() However it's hard for me to tell if I can't see you code, do you mind sending me a screenshot? you can reach me at mariyasha888@gmail.com :)
@hiseggcelency4 жыл бұрын
@@PythonSimplified Thanks for the fast response! I'll send you the code! I do have browser.get_current_page() browser.select_form() above it since i followed your guide, thanks!