Python Selenium Tutorial #1 - Web Scraping, Bots & Testing

[@jashjcomptutorials2723]

You, my friend, will go down in history as a legend

[@priyanshusaraf9495]

He has already...

[@mayurkalsekar524]

What if he defeats death and never goes down

[@ahmadhadwan]

@@mayurkalsekar524 creates a script to cheat death XD

[@mohammedsaad3801]

Haha so true

[@AbhijeetSinghs]

That’s way too big words for just a KZbin video

[@rohansylvain1702]

I was just getting into web scraping, then you dropped this tutorial! Perfect timing.

[@AbayChannel]

Goodluck .. Try with selenium ide ..

[@arkwright3274]

same with me

[@Nexus-rt1bm]

same

[@shaneebkottakkal3491]

same

[@rmt3589]

Same, was looking into Beautiful Soup.

[@kosamati9661]

Sunday after lunch (in Poland) I was just about to sit down to study and here's a new tutorial. Perfect timing

[@wojciechsuchecki5690]

Widzę w wybornych tematach się obracasz

[@twobyfour2468]

@@wojciechsuchecki5690 A jak

[@trol9880]

witam rodaków

[@weteach3263]

No cześć wszystkim :D

[@aadichauhan947]

@@twobyfour2468 aij ies dàk rozejskï

[@chrisvaia9469]

I've never watched a so clear tutorial. Fantastic tutorial!

[@kiranjoshi6721]

I've used Selenium for automating multiple routine tasks at my work. It is saving lots of time for me. Thanks for this series Tim, it'll be good brush up for me.

[@MrQuay03]

And what are those tasks?

[@usr829]

Hi kiran, i tried this but not work. Please help me.

[@pro-socialsociopath769]

​@@MrQuay03 His safety training ;)

[@snavellet5612]

My God, thank you for making a series in selenium

[@jamesgalantino2901]

Clear, concise. Great stuff, thanks so much. It's dev's like you on KZbin and more that [literally] make the world go 'round. Appreciate this series!

[@YouCantBringThemBack]

Here I am again, 4 years later, thanks for posting this 4 years ago! I knew I'd eventually forget and needed an awesome refresher course!

[@santhoshxd]

Bro you are great, I thought of making a selenium project today and you started a new series on it. Love you a lot !!

[@sapemi08]

I am an old guy, but I enjoy learning with your videos. As the time passes by I see you growing from an intelligent, agile teen to a more seasoned young man. Cheers!

[@soviet_onion_4208]

awesone tim that was a great tutorial, i like how u ran through everything that couldve went wrong and showed people how to fix it

[@ryanle6442]

This is a very detailed tutorial on intro to web scraping using Python, well done Tim!

[@geek-ko]

Time ago i was starting to understand Slenium! and now we have your selenium tutorial!! YEEES!

[@datagonia]

Hey Tim, might be a good idea to update this video, Selenium now uses Service objects to run the Drivers.

[@NewCyZ]

Ty

[@JTJaniceTek]

how can i download the chrome driver now??@@NewCyZ

[@jawadmansoor6064]

I think you are the best teacher for python. I learned a lot from your beginner's python course (4hours) and now I hope to learn more from this course. Thank you.

[@joshiashish1989]

Spent two hours on Friday evening to find solution to initial setup. I can have a cold beer now. Thanks. You rock!

[@aiamcharina]

Started learning selenium python today and this is really helpful, simple and easy to learn. Thanks a lot!

[@marius35mm]

Did you knew python before? Asking for a friend...

[@aiamcharina]

@@marius35mm Yes, a little bit. Just the basics.

[@mitchdask]

What i was trying to setup for 2+ hours with that chromedriver,solved in 5 min effort ,with this video (rookie over here!)That made my day!!!

[@sameehanif9192]

Yesterday I was searching for selenium tutorials for so long and today he posts this video

[@joshuam2341]

Happy days

[@ristoshikongo7730]

My man. A million subscribers. It's been a long time coming. Keep up the great work.

[@kanhasingh728]

What a timing man. I was just thinking of starting web scraping 😁 Great video btw

[@adrico11]

Same here !!

[@didedoshka]

Same!

[@kanhasingh728]

Glad to hear guys🙂

[@toshibiswas3115]

the new ps5 i would think

[@kanhasingh728]

@@toshibiswas3115 what?

[@abdohm809]

If the selenium version you are using is v4.6.0 or above (which I think it is as I see SeleniumManger in the error trace), then you don't really have to set the driver.exe path. Selenium can handle the browser and drivers by itself. So your code can be simplified as below: from selenium import webdriver driver = webdriver.Chrome()

[@tony6709]

Thank you so much! I was struggling with the error until I saw your comment!

[@fitrianilamsari4050]

thank u, i was struggling til i saw ur comment!

[@TheScuds]

Will this work with Firefox as well?

[@wahidurrahman7506]

I was literally about to make that same comment here after struggling for an hour

[@husseinsfitness5434]

legend, thanks

[@blossomkalabaw7941]

this is really great Tim, thanks a lot, very concise and up to details, less garbage talks, very focused on the topic. thanks

[@alfredxavier6568]

This would be a great series your fans would watch it.

[@muhammadarsalan7996]

I was a chemical engineer an year back .... Now I am approaching an MS in CS ...thanks dude ....learnt a lot from you

[@joshuam2341]

Good luck!

[@PaperFc]

Just curious, but why did you make a switch to CS from chem engineering?

[@karandewani6614]

I just made a tinder auto swiping bot yesterday and was thinking to learn more about selenium and this video pops up. Great timing and awesome videos man! Keep up the good work.

[@jammincoder]

Great explanations, easy to follow, just what I needed to get started. Definitely earned my subscription. Thanks man.

[@danielchoy7030]

Bruh your channel is omnipresent.

[@TyroneSmithTDS]

I've been waiting for this tutorial from you for so long. Thanks Timmmm!!!!

[@moresmartsolutions9480]

We have been waiting for this, finally!!!! So thanks Sir Let tech with Tim

[@ofi2187]

So clear so clean. Thank you sir, there is no one else who explains it as well as you.

[@powerb_i]

Love your videos Tim, so well presented. Thank you you're really helping me as a newbie Python programmer.

[@0Sareena0]

Better than a Udemy course. What a king you are

[@SkyFly19853]

Oh... wow... Never thought I would see Selenium on this channel... Wow...

[@tt4kcoding844]

the best python teacher in the world 💓💓

[@akberkhan6329]

Awesome man, thanks for the vid! Took me a while to figure out why I was getting an error message in my terminal...It's super important that the PATH = "/Applications/chromedriver" (this is where it was for me) for Mac user is written correctly, otherwise it won't be able to find the chromedriver ... Anyways great video, will be watching the other vids.. nice work

[@loralora8395]

very important indeed. Took me ages but felt so relieved in the end

[@karmendurbin]

It's been taking me forever to find this out too! LOL

[@mutlucaniltir]

This was the best tutorial I've seen on youtube so far. Thank you very much. Hope you get what you deserve with your effort.

[@henrygutetama3173]

FYI: For the Chrome driver on Windows, you can use chromedriver-binary library "pip install chromedriver-binary" in case you have trouble with the dowloaded driver.

[@jaredgarbo3679]

@T Bone Have you fixed the issue?

[@motlatsimoea]

@@jaredgarbo3679 I'm also having this same issue incase you managed to fix it..

[@aniketshinde4416]

So concise and to the point. Makes it look easy.

[@alpixfere]

right after i finished the course, i wrote a programm that will spam join a kahoot game that crashes game whenever teacher gives out the code, and i'll let my programm pick some inappropreate name, anyway love your courses

[@ramizquliyev394]

Man!!! You are a legend. Actually I already downloaded selenium and webdriver but You have answered all problems that I was straglling when I was trying to download. You deserve more than subscribtion and likes

[@mreverything4663]

For this tutorial series, could we build some sort of notification system that uses web scraping?

[@omgcyanide4642]

Yes I made notifications straight to my phone with spontit or discord and use web scraping you can even permanently change variables with txt files and other cool stuff

[@SquidBeats]

Jesus Christ is God and is the only way. Hell is real whether you believe it or not.

[@halamadrid5238]

@@omgcyanide4642 how did you do this?

[@barabara9855]

@@SquidBeats somebody successfully made a bot

[@ZbigniewLoboda]

❤ I love it. Unlike other tutorial authors, this one was well-prepared for the presentation.

[@nerdtowncity]

I’ll definitely be following these videos! Will help with my work!!

[@franciscolucarini8761]

Me too lol

[@0myjoe]

thank you so much. This is about the only tutorial I cant find out there that makes sense and goes through each move step by step. Defo gonna keep coming back to this channel

[@ДмитрийЕвстратов-э3м]

This is a top tier tutorial series! Thank you a lot, you're a legend dude!

[@mdemr7391]

If your tab closes automatically after you run the code and you are using VS Code, Try to run the code right-clicking anywhere in the editor and select Run Python File in Terminal. I was running the code pressing F5, I tried this and it worked

[@mkianzolz998]

thank u

[@joewidger7954]

Thank you :)

[@mehmetemin7932]

thank you

[@HallofWisdom]

--- Thank you, M de mr - I was having that very problem and now it's solved ! --- 20NOV22

[@alex17k18]

same here, but tab keeps quitting...

[@SOMEGUY-ci8nr]

finally someone that knows what they are talkin about i been searching for ever for a video like this :)

[@akremgomri9085]

I enjoyed the video so much, but there is an update for newer versions of chrome. You shoould do this : from selenium import webdriver PATH = "C:\\Program Files\\chromedriver-win64\\chromedriver.exe" cService = webdriver.ChromeService(executable_path=PATH) driver = webdriver.Chrome(service = cService) Instead of passing the executable_path to Chrome() directly.

[@dekisatria1233]

thanks, it help a lot. the chrome web driver change a lot. i think tim need to rework this tutorial

[@CSEngineerInsider]

Best video in whole KZbin.

[@akshatsingh7286]

Great Video man!! Please do make more videos on this! Also make a video on how to setup Sublime text

[@MrMarkgyuro]

big thanks, it is really good to see that you explain everything clearly and step are not left out : )

[@MrJcc444]

this has some potential but no matter what I tried it doesn't work , tried it on the mac .windows same issue 'str' has no capabilities and nowhere to go from there

[@deepakmalhotra2880]

Great video which needs to go viral. The power to do this stuff is pretty sick

[@2011Filmer]

Huge fan! Keep up the good work :)

[@andrewv8548]

Selenium is insanely powerful. Super fun to use.

[@donglinjin4765]

I just started learning Selenium and the first problem I encountered was that selenium will automatically close the window if no action is made. So basically if you run the code in this video the window will stay opened for like 2 seconds then it will be shutdown. However I found a solution, add "while True: pass" in the end of the script. I hope it helps!

[@iaretzeee]

this helped me, thank u 👍

[@thiagoabsc]

Just great, man ! Very easy and uncomplicated! Congrats! Keep up! I hope KZbin had more of so useful stuff like this!

[@dokwonsuh]

Hey Tim. Great videos series! I am really learning a lot - thank you so much. One idea for a future video might be how to set up headless Selenium on a remote server :)

[@dillonmears6696]

I work in education and the quality of your tutorials is better than 90% of the teachers I have worked with! Keep up the awesome work!

[@_Xyr]

I was looking for a selenium tutorial and couldn't find one that was atleast a year ago, Please upload the next parts asap Also amazing video, Keep up the good work

[@codewithshajahan9997]

informative and amazing as always.Thank you Tim for these tutorials!

[@vineethrao60]

if your code dosnt work put an r before the string in path for example- r"C:\Program Files (x86)"

[@ManInBlack718]

Thanks, but why does this work?

[@ManInBlack718]

@@williamanstett5305 thanks!

[@apotatowithfingers1572]

do you know how to get the path to a file on a chromebook?

[@ryanfernando8330]

I found this immensely helpful, I used to have to put double slashes in the file path every time.

[@manbearpig4702]

legend

[@programming_hut]

Could you please start a new series after this where you will teach Basic to advance projects with ML in py Btw thanks for such a good content

[@pawandeep313]

It would be the best day ;D

[@computedprogramming4705]

yes also agree with your statement

[@angel-ig]

kzbin.info/www/bejne/jXfVY4qdg56Srqc

[@ekrem_dincel]

Look at Sentdex channel

[@alex17k18]

I tried it like you did here 9:30, but why does it not show the chrome window when I run the code, instead the chrome opens for a sec and quits... I did everything right.

[@victordvickie]

I was expecting this tut

[@markbrown5365]

"Chromedriver needs to be in PATH" ERROR keeps poping up for me. I already added them to my system environment. Still the same error. Any advice?

[@Ohamskillua]

Bro copy the path, from your filemanager, and do // instead of /.

[@uskumru3106]

Try this: PATH=(r"C:\Program Files\dowload/chromedriver.exe") Basically PATH=(r"file_location/file_name.exe")

[@mananmalhotra792]

Use r’filelocation\\filename’

[@markbrown5365]

@@mananmalhotra792 thank you for answering my post. What does the "r" function do exactly & why do dome coding tutorials like this one not have them? It's confusing. When should/ shouldn't I use it?

[@jimybobjim]

@@markbrown5365 for example, if I wanted the string ‘something ’ it would create the string ‘something’, but it would have a new line at the end. If instead I did r’something ’, it would just be ‘something ’.

[@preetirani8272]

can someone please share the link for download the chrome driver for chrome version 118x

[@king_canutooo]

I learnt python from you . Thank you very much Tim .

[@vazp3]

May 2024: Following the instructions and using " from selenium import webdriver service = webdriver.ChromeService(executable_path = 'C:\Program Files (x86)\chromedriver.exe') driver = webdriver.Chrome(service=service) " worked for me, otherwise an error appears: " 'str' object has no attribute 'capabilities' "

[@sjsjakskass]

it shows unicode error smthng and i fixed it by adding (path= r'C:\Users .....the path ..')

[@bryanwelch766]

Selenium setup quickly and clearly explained. Thank you!

[@lawrencedoliveiro9104]

1:48 Debian includes standard packages for python3-selenium, chromium-driver and phantomjs. Both the latter two are listed as dependencies for the first one.

[@santoryu8940]

Hello everyone! I am getting this error: "AttributeError: 'str' object has no attribute 'capabilities' ". Has anyone encountered the same error message? If so, may I ask how did you fix it?

[@MissGurung_AI]

replace driver = webdriver.Chrome("PATH") with driver = webdriver.Chrome()

[@НиколаИванов-ю4х]

​@@MissGurung_AI Had the same error. Thank you

[@gardengnomefan]

@@MissGurung_AI I've also had the same issue, but when I do that I get an error "Error getting version of chromedriver 115. Retrying with chromedriver 114 (attempt 1/5)". And it opens like a debug? version of chrome, while I'd want it to open with my regular chrome browser. Any ideas? I'm running chrome *.171, but the chromedriver I downloaded id *.170. Is that the issue?

[@NishantSingh-px3jm]

You can install chrome driver by using "chocolaty" by the command " choco install chromedriver" it will install the chrome driver if you have chocolaty installeed and then you will not have to add the path each time. The code will be as:- from selenium import webdriver driver = webdriver.Chrome() driver.get("google.com")

[@ownzuall]

If you want to completely automate the process, chocolaty have sample of script to download it either by powershell or cmd, so you can make a cmd script that downloads chocolaty and then tells it to download the next thing.

[@NishantSingh-px3jm]

@@ownzuall yep I know but I that that would be a bit difficult for someone who has no experience in batch files

[@Lerklinten]

... actually had some issues setting up selenium, this little video helped me, so Thanks..

[@leoguitar2058]

Linux/MacOS: pip = install modules on python 2 pip3 = install modules on python 3

[@masondigiorgio7759]

Leonardo Neute If your python is up to date pip would also be python 3, since python 2 has been phased out. At least this is the case on my Linux machine. Also you can run pip -V to make sure you are using the correct version.

[@leoguitar2058]

​ Mason DiGiorgio in that case you should have only one version of python installed, using ubuntu

[@brianscalabrine6481]

when i try to do import selenium, it won’t work. import shows up as text not a command. help

[@astridzeneca1076]

got the same struggle

[@alextorres9750]

Wow...just amazing...and finally proper English spoken :)

[@marjotoska]

Heads up, in 2024, doing: "driver = webdriver.Chrome(PATH)" will throw an error. It can be fixed by removing PATH, like this: "driver = webdriver.Chrome()"

[@iaretzeee]

thank u brother this worked 👍

[@sreeharshams905]

Thank you!

[@luisemilioogando]

Even Though is an old version o selenium, Im impressed with the way you teach.. Great Job 👏

[@LukmanulHakim1999]

need this :V keep upload

[@callme_xoroz]

msg = f"Unable to obtain driver for {options.capabilities['browserName']} using Selenium Manager." ^^^^^^^^^^^^^^^^^^^^ AttributeError: 'str' object has no attribute 'capabilities'

[@elishabt662]

Tried the same and i got this error: 12296:26672:0420/163936.459:ERROR:browser_switcher_service.cc(238) XXX Init() Any help would b of great use

[@truog91]

i got this too

[@abhilashtu3653]

Hi Tim thank you for your help And I have a question Why sometimes selenium act weird, like sometimes the send keys and click method will not work even if I have no errors. This is a huge strange problem it not true in every case if I had a script which not working because of the send keys issue but I copy the code and paste in a another file it worked , why is this?

[@redbutton_photo]

I ran into something similar. Rerun PIP import selenium. Also make sure you're running import selenium from webdriver

[@aniket7512]

If chrome closes immediately after launching then add this to the code at end while(True): pass

[@NourSoltani-mh1xh]

For those who watch this video years after it posted. I think a few things are changed: 1) you do not need to download the web driver anymore. Just use driver = webdriver.Chrome() 2) if you open a page and it closes immediately, add the following (assuming you are using Chrome as well): from selenium.webdriver.chrome.options import Options options = Options() options.add_experimental_option("detach", True) driver = webdriver.Chrome(options= options) These are two main issues I had when I was coding along. Took me sometime to figure out a solution. Hope it helps.

[@AnthonHax]

Nice tutorial! I used geckodriver and it works fine too; but path/'driver' variables must be different with gecko: driver = webdriver.Firefox(executable_path=r'PATH\geckodriver.exe') took me a while to find out, so I'll just leave it here.

[@btraboulsi]

It was driving me crazy hahahaha. Thanks for the tip!

[@user-du8hf3he7r]

Update for Selenium4: Replace: driver = webdriver.Chrome(PATH) With: from selenium.webdriver.chrome.service import Service service = Service(PATH) options = webdriver.ChromeOptions() driver = webdriver.Chrome(service=service, options=options)

[@پارساوالیزادهاردلان]

Thanks for your advice.

[@friendlygodzilla]

You know you can not include the path, right? driver = webdriver.Chrome()

[@user-du8hf3he7r]

@@friendlygodzilla Oops. My mistake.

[@nujranujranujra]

If anybody is getting the error:SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape Add an 'r' before the path PATH = r"C:\"

[@thelifeofsarath]

Thank you!

[@tobortwo3697]

still didn't help

[@tobortwo3697]

@GucciVittion thanks, that's helpful. it wasn't my problem actually, but I got it fixed so it's okay. (solution: add executable path=r before the driver, but not for PATH)

[@k2icc]

WebDriverException: Message: chrome not reachable

[@lovely-shrubbery8578]

I couldn't find the particular exception, but I'd look through this selenium-python.readthedocs.io/api.html#module-selenium.webdriver.chrome.webdriver if no luck I'd try the Firefox driver

[@ymcho4037]

what is your PATH

[@k2icc]

@@lovely-shrubbery8578 I believe that I used another path withing a folder, so that worked out fine. Now, I get empty results when scraping from tradingeconomics.com/commodities tables. A little progress, but now if arrays, those results come back empty. If anyone can try getting data from those tables, what will be great. Also tried bs4 to get into the classes of td, tr, etc. No results.. Thanks.

[@herrdesai6853]

having error as in "chromedriver.exe"

[@akittensjournal]

Same here . Your's resolved?

[@ronkzedonk]

@@akittensjournal check the description (second link)

[@serene9228]

same

[@katharinaprzeperska5624]

@@ronkzedonk The description of what? Chrome driver? Nothing useful in there or I can't see it. How did you solve the error?

[@pickledparsleyparty]

Real talk: Is there anyone out there who's advanced enough to want to use Selenium but basic enough that they don't know what Pip is nor how to install Python packages?

[@loganlogan2411]

no

[@yay69]

me(?

[@mr2_mike]

Yep!

[@doomstarks182]

Yeah so I know pip is what you use to install python packages? But it’s real easy for me to forget and I actually had to look up what it meant bc I was having problems with pathing and needed to fix. Installing the packages are easy but i had to to learn about relative and absolute paths and I watched a video and learned that my files locations were all jacked up and the video fixed that issue for me. I’m learning on my own and at my new job we do some data entry for new hires. We have to get id copy paste from one form to another. One of the big things is when the forms don’t match it kicks an error report and we manually go through looking for the mismatch. We have to get the new hires to fill out a paper and then we trial by error find the mistake. My first day I was like woah this is tedious and time consuming I bet I could find a way to get the information from the forms compare them and where they don’t match have the correct info put into the form. So here I am learning how to do that :)

[@obamacheeseburger2046]

Basically me

[@jampavy6446]

Really whatever this man teaches sits in my brain very nicely. Thank you so much and I like your vids very much.

[@isabelalvarez4467]

This video is gold! I have needed to scrape in two different computers and i´ve come back to install everything, cause its so well explained. pls keep it online forever (L)

[@TheAraz95]

Nice, I was suggesting you to make a selenium tutorial. It's really fun for newbies!

[@cass_666]

Short and sweet, you always have excellent videos.

[@gabymaymo8482]

This is the tutorial i was looking for, very clear, you have a new subsc

[@ankushsarkar1746]

Holly shit dude , I discovered selenium like 3 days back and u maikin this. Amazing man

[@lod7635]

this gives possibility in infinite amount of things, thank you so much.

[@rolandlefranc9087]

You made me save some much time with your great short tutorial. Thank you!

[@multigladiator384]

Thanks a lot! That was exactly what I was looking for. Looking forward to some High Quality content on your channel. Subscribed.