Excel VBA Introduction Part 57.2 - Basic Web Scraping with Selenium and Google Chrome

[@walkedintoawall4893]

The hidden gem of KZbin ! Andrew deserves millions of subscribers! So glad I found you. Please keep on providing awesome tutorials and spreading amazing knowledge to the world !!!

[@WiseOwlTutorials]

Thank you! I really appreciate the support, thank you for taking the time to leave a comment!

[@laiwong221]

I second this!!!

[@blakepeterson1977]

FYI - I was able to overcome the Chromedriver version error by saving the downloaded version, per the error message, to C:\Users\(username)\AppData\Local\Selenium Basic\Chromedriver.exe. I tried various other solutions to no avail. Thanks for a great video! I'm overdue updating my Internet Explorer web scrapers. This is a big help.

[@WiseOwlTutorials]

Happy to hear that you solved it Blake, thanks for taking the time to share your solution!

[@tg5441]

I have watched 2 times, the first time is typing the code when the video is proceeding and 2nd time is reviewing and digesting the code. Thanks for the valuable video.

[@WiseOwlTutorials]

You're very welcome! Happy to hear that you found it useful!

[@a_stray_dog]

WiseOwl! Just wanted to take a minute to show my appreciation for this video. I've been following it for the last few days as I build my own web-scrapper to automate a rather lengthy procedure I do daily as part of my workflow. Originally I was planning on creating a separate web-scrapper with Python that I would then have to format using macros, but thanks to you, I'm able to store the whole process neatly into a single workbook! Plus, I learned a whole lot of new stuff about VBA along the way. You're a fantastic and incredibly engaging teacher, god bless!

[@WiseOwlTutorials]

Thanks for taking the time to write this, it's great to hear that the videos have been so useful!

[@excluzive.]

Andrew, your videos are just priceless. I watched a lot of them and It gave me sooooo much when I started to study VBA. I am so grateful to you! May the Force be with you!

[@WiseOwlTutorials]

Very happy to hear that you've found the videos so useful! Thanks for the comment (and the SW reference!)

[@tuyoexcelypowerbi]

Honestly, I think you're the best instructor in KZbin

[@WiseOwlTutorials]

Thank you so much, it's very kind of you to say!

[@octavioilachoqueollachica203]

👋

[@fuatarslan3359]

@@WiseOwlTutorials Dear Valuable Content Creator, I have been following your content for a short while. I think your content is in the standards of training sets. By the way, I am trying to reach you from Turkey. English is not known enough in Turkey. For this reason, I would like to voice your content in Turkish and ensure that it is delivered to your followers in Turkey in a way that will be more beneficial. Of course, I will need your permission for this. I will redirect you to your youtube account and website for the original content link under each of your content that I voiced. As you know, it does not make a lot of money for its producers on youtube for such specific issues. The purpose of this proposal is for non-English speaking enthusiasts to get to know you and better understand what you are telling. I hope you welcome my offer. I wish you good work. Fuat Arslan

[@WiseOwlTutorials]

@@fuatarslan3359 Hi Fuat, many thanks for your kind offer! I'm only an employee of Wise Owl and all of the video content is owned by the company. I'm sorry but the company won't give permission for others to make copies of their content.

[@italochiarabini9911]

Thank you!! With this video you saved my 10 years work building a VBA automated scraper system based on IE. Over 10k lines of code! I look forward Microsoft great thinkers get what they deserve, as they got with Windows 8.

[@WiseOwlTutorials]

Happy to hear that it helped Italo! Your system sounds incredible!

[@dlwndrms21]

You have relieved me of a huge burden. In particular, it was easy to follow the flow of your lecture because it was explained using one web site and it could be applied immediately because you showed representative cases. Thanks

[@WiseOwlTutorials]

Happy to hear that the video helped you, thanks for watching!

[@gaganrastogi9624]

Your videos are the best. I love the way you teach 😊. I never felt that your videos are too long because of your teaching style. wise owl 🦉 Is One of my favorite channels on KZbin. I can feel the hardwork, you put into every single video. Like adding chapter, or explaining code in simple words. I really like your efforts from my heart.

[@WiseOwlTutorials]

Thank you Gagan, I really appreciate the comments!

[@AI-ec2qb]

I don't want VBA to disappear 😭 Thank you Google for keeping VB alive

[@AI-ec2qb]

WOW! A Real Excel Máster! I watch for 1 hour straight! Best Tutor! Respect!

[@WiseOwlTutorials]

Thank you, glad you enjoyed it!

[@arvinthnair2856]

Your videos have been excellent. Although i have a finance background, your videos helped me automate my work for my team. I got tons of praise for it. The VBA for internet explorer used to work and since it's being decommissioned, i would need to amend the code for Chrome. I could not find any material elsewhere but the best place to find it is your channel. Thanks Andrew! & i made a donation as well

[@WiseOwlTutorials]

Thank you Arvinth! I appreciate the support and am happy to hear that the videos have helped you so much!

[@ComercioExteriorUV]

Hi Andrew, your videos have helped me so much, they have taken my career to another level :) Before I was being flooded by Excel reports, and now with Macros I do the same job with a couple of clicks, And this allows me to have time to learn even more, Thanks again,

[@WiseOwlTutorials]

Very happy to hear that Mauricio! Thank you for watching and for taking the time to leave such a nice comment!

[@bencole8301]

I literally could not have asked for a more specifically relevant video needed than this one I have just watched and completed if I had actually paid for it. It was quite simply perfect. This is all thanks to (in my opinion) one of the very best tutors I could have ever wished for in Andrew Gould. You make me feel about VBA and coding as Yoda made Luke feel about being a Jedi. Do or do not. There is no try.

[@WiseOwlTutorials]

Ben, I love this comment! Genuinely happy that you found the video so useful and I'm grateful that you took the time to pass on your feedback. This channel would be nothing without an audience of enthusiastic people like yourself - happy to have you on board!

[@Fly0High]

Uff My poor little excel dedicated computer took a while to scrape all 130 results on "excel VBA" Thank you so much! On to the Task!

[@WiseOwlTutorials]

I hope your computer survives!

[@jacundill]

Absolutely fantastic, these videos are so easy to understand and follow, already thinking of the many ways to utilise the skills and knowledge learned in this particular video. Big thanks!

[@WiseOwlTutorials]

Happy to hear you've found it useful, thanks for watching!

[@Bulbasteeb]

best best best channel. It has increased my workflow and allow to develop tools for my department

[@WiseOwlTutorials]

That's great to hear Stephen! I'm happy that you've found the videos useful, thanks for taking the time to leave a comment!

[@robinsparkles]

the way you teach is AMAZING it’s so hard to find a good tutorial on vba thank u so much i hope u share more about vba :) i love u

[@WiseOwlTutorials]

Thanks so much!

[@mkhalil92]

I must say this is the best web scrapping video for beginners. Thanks a lot

[@WiseOwlTutorials]

I'm really happy to hear that you found it useful, thank you for watching and for taking the time to leave a comment!

[@metalworker3]

Excel-lent video with easy to follow process logic. Thanks!!

[@WiseOwlTutorials]

😀 glad you enjoyed it!

[@bk3461]

Wow! Thank you Andrew! This is by far the best tutorial I've come across online on this topic!!

[@WiseOwlTutorials]

Happy to hear that you found it useful! Thanks for watching and taking the time to leave a comment!

[@ExcelForFreelancers]

Very well done. Thanks so much for sharing

[@WiseOwlTutorials]

You're welcome, thanks for watching!

[@triptitiwari3663]

Hi Andrew, Thank you for posting this tutorial. This tutorial explains everything. Your videos are very educative and informative. Your video is the far most best video on web scraping with selenium and google chrome. I was looking something on this topic from a very long time.

[@WiseOwlTutorials]

I'm so happy that you found it useful! Thank you for watching and for taking the time to leave a nice comment, we appreciate it!

[@cscs3818]

Hello Andrew! I have been watching some of your content and decided to finally comment to say thank you for your awesome work. Your content is very high quality and you are amazing teacher. I wish you all the best from Hungary!

[@WiseOwlTutorials]

Hello! Thanks for watching and for taking the time to write a comment!

[@sacredchuck]

I am extremely thankful for your helpful videos Andrew! You're the man!

[@WiseOwlTutorials]

Thanks Charlie!

[@pbassociates9168]

Wonderful, structured and very informative tutorial, sir I have seen your all 102 tutorials they are very good thank you for sharing you precious knowledge.

[@WiseOwlTutorials]

You are very welcome! Thank you for watching and taking the time to leave a comment!

[@jmathew6988]

Comprehensive tutorial as always. Thank you sir.

[@WiseOwlTutorials]

Thank you, glad you found it useful and thanks for watching!

[@jarguedas]

Your videos are amazing, thanks, greetings from Costa Rica

[@WiseOwlTutorials]

Thank you Jorge!

[@vijayalakshmibadigeru4479]

Very useful video.thanks, Looking forward for more advanced videos like ms access VBA, ms word VBA.👍

[@WiseOwlTutorials]

Thank you Vijaylakshmi!

[@MerryNose]

brilliant! Thank you so much!

[@WiseOwlTutorials]

Thank you for watching!

[@НиколяБухталин]

You are great! And I wish you the very best today!

[@WiseOwlTutorials]

Thank you! And all the best to you as well!

[@stephennicholson8135]

Thanks. As usual a very clear explanation of a complicated topic. :)

[@WiseOwlTutorials]

Thanks Stephen, glad you found it useful and thanks for watching!

[@Survival_Guides_313]

I used the internet explorer browser to scrape from Google Maps, it does not load more than 10 results, after finishing my code with the conventional VBA libraries (not Selenium) , checked the same link in a chrome browser and discovered that it gathers all results possible, so searched for automation using Selenium and found this playlist, it is awsome and specially this video where you can check if the element is there or not as in Google maps results you may find a website or not, a phone number or not and so on, Awsome videos till now

[@marcelorebel]

Your classes are amazings! Thanks for all

[@WiseOwlTutorials]

Thank you Marcelo!

[@MoraTara]

Hello, thank you for the video. I need help, please What should we do? If the site requires you to log in first, then scrape the data from it after that

[@williamkobertz3386]

If I design a macro-enabled Excel file using Selenium to do web scraping per your video, will the file work on another computer that only has Chrome installed? Essentially, I want the file to do automated work for folks in my organization that are not VBA- nor computer savvy. I started to create one using IE, but didn't know about the sunsetting to Microsoft Edge.... Thanks in advance! Best, Bill

[@WiseOwlTutorials]

Hi Bill! Unfortunately, you would need to install the SeleniumBasic library on the computers which run the code (and keep the ChromeDriver up to date on those machines as well). It's not an ideal solution but unfortunately there aren't any good solutions for distributing VBA-based web scraping projects.

[@williamkobertz3386]

Thanks for the reply! Good to know about updating the ChromeDriver.@@WiseOwlTutorials

[@JK-hm3lf]

WoW. What a detailed informative video. great helping content.

[@WiseOwlTutorials]

Thanks JK, glad you found it useful!

[@KhalilYasser]

Thank you very much my tutor Andrew. Really amazing and awesome. I know a lot about selenium but I have enjoyed watching your video and the video added new points to me.

[@WiseOwlTutorials]

Thank you Yasser! Please feel free to point out any mistakes that you see!

[@KhalilYasser]

@@WiseOwlTutorials No mistakes. You are simply PERFECT.

[@WiseOwlTutorials]

@@KhalilYasser :D thanks!

[@mrindian7422]

This is at another level thanks.....

[@WiseOwlTutorials]

Thanks, glad you liked it!

[@jerryto9010]

Thank you Andrew, just started my Selenium tour with your videos, but I've one question: is it possible to get the "xpath" by using contains.text function? I'm trying to retrieve all text from every post of facebook, looking for keywords and then capture all texts / links/ and the name of the poster, please help! Thank you!

[@WiseOwlTutorials]

Hi Jerry! You'll find a bit more information about XPath in part 57.4 of this series, I hope it helps!

[@DM-py7pj]

Excellent tutorial as usual and I love referring people here. Your skill as a teacher is wonderful. Hopefully you will cover more on the power of using css and xpath e.g. 30:13 avoid loop with xpath e.g. "//h2[contains(text(), 'Video tutorials (')]" or "//h2[contains(text(), 'Video tutorials (')]/following-sibling::*//table" ; modern browsers are optimized for css selectors so it is an essential skill in the webscraper's toolbox. I appreciate it becomes a lot more in depth at that point but perhaps you do a couple of short videos just outlining those? FYI To get the links included you could instead transfer the outerHTML of the table element to clipboard and paste to sheet. Few more lines of code however. Very much looking forward to Dax measures videos!

[@WiseOwlTutorials]

Thank you so much! And yes, I'll be covering how to find by css and xpath later on (I thought this video was long enough already without including those!). Thanks for the tip about the clipboard, I'd forgotten about that. I appreciate all the comments and thanks for watching!

[@DM-py7pj]

@@WiseOwlTutorials Love your videos!

[@Damned943]

Thanks for a thousandth time!

[@WiseOwlTutorials]

You're very welcome, thanks for watching!

[@phioj1886]

Thanks Andrew! This is the best!!!

[@WiseOwlTutorials]

Cheers phio, glad you liked it!

[@konradstrozak9965]

Hello Andrew, Your videos are fantastic, love the passion in your voice. Actually i have one question and hope that you will help me even though this video is 2y old. But when i put in vba the formula ....FindElementByClass(...).Click i have a debug which says "Run - time error "32" : Invalid selector Error invalid selector From java script error {"status":32 " value : "Compund class names not permitted"} (sessio info: Microsoft Edge = 126.0.2592.87) Driver info: msedgedriver = 126.0.2592.87 and some random numbers and letters.platform=Windows. Do you know maybe hot to repair it ? Thank you for the help

[@WiseOwlTutorials]

Hi! You might find this video helpful kzbin.info/www/bejne/oqOadHmQeq5lj6Mfeature=shared I'm fairly certain I mention compound classes in there and it provides various other options for finding elements that you should find useful. I hope it helps!

[@abdollahfargo9401]

Thank you so much for sharing With us that much informations

[@WiseOwlTutorials]

You're very welcome, thank you for watching!

[@lakshyatashrivastava480]

This is very helpful. I just want to ask to how one can learn to know the amount you know?

[@WiseOwlTutorials]

Many years of practice!

[@krn14242]

Thanks Andrew. Great video.

[@WiseOwlTutorials]

You're welcome Kevin, thanks for watching!

[@janezklun]

Thank you Andrew, for great lesson, the best, Cheers

[@WiseOwlTutorials]

You're welcome Janez, happy to hear that you enjoyed it!

[@DStudpuppy]

Andrew, this is an awesome video. However I am having difficulty with the current website and at 11:50, you are FindElementByName("what"). On the current website there is a question mark, and I cannot figure out how to navigate the input of "Excel VBA" to that dropdown. How do I do this with the current website? Many thanks

[@WiseOwlTutorials]

Indeed, the problem with websites is that they're constantly changing! You might be able to find an old version of the Wise Owl website on the Wayback Machine which you could use to practise - here's one that might work web.archive.org/web/20221228132939/www.wiseowl.co.uk/ Hope it helps!

[@jimhubbard7075]

Simply fantastic!

[@WiseOwlTutorials]

Thanks Jim!

[@ianpitcher8252]

great video. I am still having trouble holding the Website up, its still closing down at the end of the routine, even though l put the private ch as Selenium. Chrome Driver

[@WiseOwlTutorials]

Thanks Ian! Strange that your browser closes. Providing the variable remains in scope at the end of the procedure and you haven't explicitly closed it, it should remain open.

[@datagru189]

Hi Andrew, you are the best teacher. The way to describe small small point that so useful & I leaned all data skill from your channel . Thanks for that. I have one question can you please how to trigger the pop window that comes after downloading some file from web. I want to save download file to specify folder. Thanks for all your efforts on teaching.

[@WiseOwlTutorials]

Thanks so much Ravi! The downloading files topic with Selenium is on my list. We do already have a video which shows how to download files using a Windows API function and XML HTTP requests which you might find useful if you haven't already seen it www.wiseowl.co.uk/vba-macros/videos/vba-scrape-websites/downloading-files/

[@mohammedsajidiftekharahmed6013]

Thanks a lot for your efforts

[@WiseOwlTutorials]

Thanks for watching!

[@hadibq]

excellent as always! TY

[@WiseOwlTutorials]

You're welcome, thanks for watching!

[@arielgonzalez2002]

Muchas gracias!! Saludos desde Chile

[@WiseOwlTutorials]

You're very welcome Ariel!

[@vicenteantoniomagallanesju5289]

THIS IS BEAUTIFUL I need to say this you are great hehehe you are the best at explaining this thanks for everything

[@WiseOwlTutorials]

Ahh thank you Vicente!

[@vicenteantoniomagallanesju5289]

​@@WiseOwlTutorials hello and thanks for your answer, i have a big question and i need your help, who can work with popups when i need to click on this popup?

[@WiseOwlTutorials]

@@vicenteantoniomagallanesju5289 Hi Vicente! It depends on the type of popup. Here's an example of a simple popup to confirm cookies settings kzbin.info/www/bejne/Y5zNopJunp6Vq68 I hope it helps!

[@vicenteantoniomagallanesju5289]

@@WiseOwlTutorials Great thanks for you answer you are great teacher

[@Eric-gz1uw]

Thank you Andrew！

[@WiseOwlTutorials]

You're welcome, thank you for watching!

[@novicerider751]

Fantastic video! You mention towards the end that there are ways to speed the process up by switching a few things off. Is it possible to turn the browser visibility off like you can do with "IE.visible = false"?

[@WiseOwlTutorials]

Thanks! And yes, you certainly can run Chrome invisibly - I have a video recorded but not edited which demonstrates this which I'll upload hopefully some time this week. While you're waiting, you could try this: Sub GetTopFilms() Dim cd As Selenium.ChromeDriver Dim t As Selenium.WebElement Set cd = New Selenium.ChromeDriver cd.AddArgument "--headless" cd.Start cd.Get "en.wikipedia.org/wiki/List_of_highest-grossing_films" Set t = cd.FindElementsByCss("table")(1) t.AsTable.ToExcel Worksheets.Add.Range("A1") cd.Quit End Sub And for more information, see here developers.google.com/web/updates/2017/04/headless-chrome Hope it helps!

[@novicerider751]

@@WiseOwlTutorials Worked like a charm, much appreciated!

[@WiseOwlTutorials]

@@novicerider751 You're welcome!

[@elricho72]

Great video,How can I do when a link that we click by code returns a new page in another tab and we want to capture that new page to continue capturing information from that new. Thanks for any help

[@WiseOwlTutorials]

Hi Richo! You might find video 57.6 in this playlist useful kzbin.info/aero/PLNIs-AWhQzcl3xKvF8sVL4sWRWICj_clM I hope it helps!

[@elricho72]

@@WiseOwlTutorials thanks Bro !

[@WiseOwlTutorials]

@@elricho72 No problem!

[@Yorumcu63]

Amazing video Thank You.I want ask this. For example I was make application like in here sample for my friend.it must installed too Sellenium library in my friend's computer too? Again thank you for this amazing tutorial

[@WiseOwlTutorials]

Yes, that's correct. Your friend will need the Selenium library installed too. I hope that helps!

[@Yorumcu63]

@@WiseOwlTutorials Thank you for reply ? Could you make an video about automatic installation example for Selenium ? I mean VBA checking Chrome version and and copy suitable Chromedrive exe to suitable folder ? Also I want ask this How can make Turkish text reading application in VBA ? I mean exce (or any installed library etc ) can talking Turkish text ? Google Documents supporting to Turkish voice API Could you make this kind video ? (Sorry for my bad English, I hope undertand me )

[@WiseOwlTutorials]

@@Yorumcu63 Your English is much better than my Turkish! I don't think the automatic installation of Selenium would work - in order to open Chrome and check the version (I'm not certain if that's possible in VBA) you'd need to have Selenium installed already. You can perform basic text-to-speech in VBA using the Application.Speech.Speak method so for example: Application.Speech.Speak Range("A1").Value would speak the contents of cell A1 in the default voice assigned to the Text-To-Speech settings in Windows options. We have a video which covers the basics of this technique which you can see here kzbin.info/www/bejne/d3WVn56KfNCCaqc Microsoft's documentation says that there is a Turkish voice and speech pack (his name is Tolga!) and you can see how to install it here support.microsoft.com/en-us/topic/how-to-download-text-to-speech-languages-for-windows-10-d5a6b612-b3ae-423f-afa5-4f6caf1ec5d3?ui=en-us&rs=en-gb&ad=gb I hope that helps!

[@officialknowledgebooster7839]

I am First..... Thank You Andrew Sir...

[@WiseOwlTutorials]

You're welcome!

[@krn14242]

Sir Andrew. He is British. 😀

[@EzciieMissyou]

Hello! I'd like to ask what should I do if the available tags are only 'Style' and 'Title'? I am trying to click the Title portion. Thank you!!

[@WiseOwlTutorials]

Hello! I think that you'll find Part 57.4 in this playlist useful kzbin.info/aero/PLNIs-AWhQzcl3xKvF8sVL4sWRWICj_clM I hope it helps!

[@piotrkuc8936]

Thank you for another great lesson! Any plans to continue C# series?

[@WiseOwlTutorials]

Thanks Piotr! I don't have immediate plans to continue the C# series I'm afraid but thank you for asking!

[@rohitsinha2209]

Thanks so much .. just want to know how to find window in web application by using selenium Google Chrome driver

[@dasRepalle]

Nice videos. Is it possible to open multiple tabs, And later navigate between tabs, using Excel VBA. If so please help me with that, how to do it in excel. Thanks in advance.

[@WiseOwlTutorials]

Hi Jawahar! Yes you can open new tabs and navigate between them. It's probably easiest to point you to some examples which other people have written, I hope that it helps! stackoverflow.com/questions/53381891/create-new-tab-for-each-google-search-by-selenium-in-excel-vba stackoverflow.com/questions/55666348/open-new-chrome-tab stackoverflow.com/questions/56314471/selenium-basic-vba-focus-on-new-tab-that-is-opened-by-click

[@dasRepalle]

@@WiseOwlTutorials thank you so much. It helped a lot.👍

[@WiseOwlTutorials]

@@dasRepalle That's great! Happy to hear that it helped and thank you for letting us know!

[@dangre00]

Andrew, thanks so much for your tutorial! Do you perhaps know how to execute a print command after running ch.Get? With the old InternetExplorer object, this was simply done by running the line Explorer.ExecWB 6, 2, "", "" but for the life of me can't figure out how to do this with the ChromeDriver. Thanks again.

[@WiseOwlTutorials]

You're very welcome, glad you enjoyed it! I have a feeling that I'm missing an easier way to do this but the simplest method I could find was to disable Chrome's print preview and then send keystrokes to simulate pressing CTRL + P followed by ENTER Sub SeleniumPrintWithoutPreview() Set cd = New Selenium.ChromeDriver cd.AddArgument "--disable-print-preview" cd.Start cd.Get "www.wiseowl.co.uk" VBA.SendKeys "^p" Application.Wait Now + TimeValue("00:00:02") VBA.SendKeys "{Enter}" End Sub If you want to work with Chrome's print preview, that's a little trickier! The print preview runs in a separate window and contains a nested set of shadow DOMs (here's some background on what that means developers.google.com/web/fundamentals/web-components/shadowdom) Here's some very inelegant code to hit the print button in Chrome's print preview: Sub SeleniumPrintWithPreview() Dim ShadowHost As Selenium.WebElement Dim ShadowRoot As Selenium.WebElement Dim PrintButton As Selenium.WebElement Set cd = New Selenium.ChromeDriver cd.Start cd.Get "www.wiseowl.co.uk" VBA.SendKeys "^p" 'switch to the Print Preview window cd.SwitchToNextWindow 'Walk down the nested shadow DOMs to get to the print button Set ShadowHost = cd.FindElementByCss("print-preview-app") Set ShadowRoot = cd.ExecuteScript( _ Script:="return arguments[0].shadowRoot", _ arguments:=ShadowHost) Set ShadowHost = ShadowRoot.FindElementByCss("print-preview-sidebar") Set ShadowRoot = cd.ExecuteScript( _ Script:="return arguments[0].shadowRoot", _ arguments:=ShadowHost) Set ShadowHost = ShadowRoot.FindElementByCss("print-preview-button-strip") Set ShadowRoot = cd.ExecuteScript( _ Script:="return arguments[0].shadowRoot", _ arguments:=ShadowHost) Set PrintButton = ShadowRoot.FindElementByCss(".action-button") PrintButton.Click End Sub I'm not sure if there's a better way to do this (surely there must be!) but I'll have to spend some time investigating. I hope that helps in the meantime!

[@dangre00]

@@WiseOwlTutorials Wow, thank you SO much, Andrew! The no print preview seems to be working just fine for my purposes. I'm actually sending web pages to PDF, so no dialog is the preference. Is there an easy line of code that suppresses the Chrome browser altogether so it runs in the background? Thanks again!

[@WiseOwlTutorials]

@@dangre00 No worries! There certainly is a way to run Chrome hidden using Headless Chrome (background info here developers.google.com/web/updates/2017/04/headless-chrome). I have a video on this but it's not yet published so here's the basics: Sub GetHighestGrossingFilmsTable() Dim cd As Selenium.ChromeDriver Dim t As Selenium.WebElement Set cd = New Selenium.ChromeDriver cd.AddArgument "--headless" cd.Start cd.Get "en.wikipedia.org/wiki/List_of_highest-grossing_films" Set t = cd.FindElementsByCss("table")(1) t.AsTable.ToExcel Worksheets.Add.Range("A1") cd.Quit End Sub Hope it helps!

[@dangre00]

@@WiseOwlTutorials You're the best, can't wait for the new video!

[@erikeringaad8638]

hi Andrew, Great edu again. Wonder if you could make one on popups, switch etc. i'm stucked there... after clicking to a new window/page but I can't acces the elements because i'm still in the page where I came from. Hope you understand Thanks

[@WiseOwlTutorials]

Hi Erik, thanks for the comment! Do you have the link to the page/pages you're having popup problems with? There are a few different types so it would be useful to see the exact one you're dealing with, thanks!

[@erikeringaad8638]

@@WiseOwlTutorials Thanks for reply. I allready foud the solution by finding the url of that new screen and followed that link I ended up in the right page and can access the elements.

[@WiseOwlTutorials]

@@erikeringaad8638 Good stuff, thanks for letting us know Erik!

[@alindumitru3670]

Hello and many tanks for this great tutorial. I found a problem during my test on the website I want to use with Excel. The problem is that at the las point of the order process, to confirm an submit the order I have a Pop-up message where I need to click OK. Unfortunately I don't now how to deal with it. Do you have a solution for this problem? Thank you in advance, Alin

[@WiseOwlTutorials]

Hi Alin, sorry I don't have a specific technique to use for this, which site are you using? If you post your code I might get chance to look at it but you might have more success posting a question at somewhere like StackOverflow.com (they have many people ready to answer questions!)

[@numistrum-sidera3051]

Fantasti! very Good! compliment!

[@WiseOwlTutorials]

Thank you!

[@fredyperez3227]

Andrew - amazing work mate, question, how best to deal with a ElementNotVisibleError element not interactable, ? this is happening as i try to click on a sub menu that sits under a parent menu item. Thanks mate.

[@WiseOwlTutorials]

Thanks Fredy! There could several reasons for this - it's possible that adding a simple Wait between clicking the parent item and clicking the sub menu item could solve it. Do you have the specific example you're trying?

[@Mundotech_oficial]

Parabens pelo tutorial

[@WiseOwlTutorials]

Thank you Mateus!

[@SpaghettiCode5]

Now that IE is retiring but you need the automation that IE offers. Fortunately, there's SeleniumBasic and can be used together with MSHTML, so all the things that I have learned to control IE from this channel over the years won't go to waste, which is a relief. Just a quick question, please: How to interact with browser's 'Save' dialog box after clicking on download button or link? Thank you very much, by the way for posting this helpful video..

[@WiseOwlTutorials]

Hi! I don't know how to interact with the Save dialog box. I'd just use the URLDownloadToFile function kzbin.info/www/bejne/gIHIq6WNrNumgK8 I hope it helps!

[@SpaghettiCode5]

@@WiseOwlTutorials Thank you very much for the link. I believe, I've seen that video. Unfortunately, I don't have a download link, the 'href=' attribute only contains "#". But if you click it, a dialog box appears. On IE, it's the yellow banner at the bottom. There is a code on SO-StackOverflow on how to capture the handle of the window using Windows API, just like the one on the link that you shared although a different .dll file. But there, it's just a code with insufficient explanation. I will just connect the idea from the link you shared and the code from SO because I think both have the same approach, just different .dll files. Thank you so much. I appreciate it..

[@WiseOwlTutorials]

@@SpaghettiCode5 I hope you get it to work!

[@danielteixeira4816]

Oow. Very good. I like this video. I'm from Brazil 🇧🇷

[@WiseOwlTutorials]

Obrigado Daniel!

[@benediktusdanangsatria4967]

Thankyouu for making this video!!

[@WiseOwlTutorials]

You're very welcome, thank you for watching it!

[@MerryNose]

Thank you!

[@WiseOwlTutorials]

Thank you Alexey!

[@joshuagleim5439]

Well done

[@WiseOwlTutorials]

Thanks Joshua!

[@faiz.ahmad65]

Thanks you for useful topic. Can u plz also make a video to select a value from drop-down list

[@WiseOwlTutorials]

You're welcome Faiz! We will have a video on that topic, yes

[@maksymaddstartup5590]

Спасибо! Очень позновательно!

[@WiseOwlTutorials]

You're welcome, thanks for watching!

[@hamidmirzaie1354]

Hi tanks for fantastic tutorials i have some problem i install latest version of firefox and selenium but when i want to start firefox have this error:"firefox failed to open the listening port 127.0.0.1:58849 within 15s

[@WiseOwlTutorials]

Hi Hamid, unfortunately SeleniumBasic no longer works with FireFox. The SeleniumBasic library was last updated in 2016 and appears unlikely to be updated to work with the changes to FireFox.

[@shiva11112]

Hi Andrew, is there any way we can listen to web events such as after navigation to particular website or back to original home page etc in VBA as we see in java or c#

[@WiseOwlTutorials]

Hi Shiva, I'm sorry I don't know the answer to that one!

[@shiva11112]

@@WiseOwlTutorials that's alright. Thanks for responding

[@jamespartington7835]

Hi Andrew - very informative thanks. I am having an issue with Sendkeys. The code comes up with an ‘element is not interactable’ error. If I manually click on the text box I and trying to enter the text in, the code will work. Any ideas?

[@WiseOwlTutorials]

Hi James! If you haven't already done so, try applying the Click method to the element first MyElement.Click MyElement.SendKeys "abc"

[@jamespartington7835]

@@WiseOwlTutorials Hi Andrew - I still get the error, but I have found that the code does work if the "inspect element" box on the right is open. Obviously this is not a solution, but does it give indication as to a potential solution?

[@WiseOwlTutorials]

@@jamespartington7835 Hi James! Is this just a case of the element not being ready in time for your code? Have you tried adding a wait between loading the page and sending keys? kzbin.info/www/bejne/n5qUfaucesuMapY I hope it's as simple a fix as that!

[@jamespartington7835]

Hi Andrew, I don’t think it is a wait issue more a display issue as when I test to see if the element is displayed using .isdisplayed, it false when the inspect element box is closed and true when it is open. It is rather puzzling!

[@WiseOwlTutorials]

@@jamespartington7835 Agreed! Sorry I'm not sure what to suggest at this point!

[@rajivs]

Superb well explained video. Many thanks! One question - is there any video where I can learn how to extract test and close alerts or pop-ups from a web page?

[@WiseOwlTutorials]

Thank you Rajiv! We don't have a specific video on closing alerts and pop-ups although I did cover one technique for pop-ups on Yahoo Finance as part of this video kzbin.info/www/bejne/lXrPnJeIrMx8rck Not every pop-up is the same however, so that technique may not work for every situation!

[@ngantor0163]

chromedriver vba excel import data csv to website. (im login user and password and import data ) is it possible to use excel?

[@williamkobertz3386]

Thanks!

[@WiseOwlTutorials]

Thanks so much Bill, really appreciate your support!

[@minipcgo]

Brilliant

[@WiseOwlTutorials]

Thanks!

[@pharehphatah8767]

thanks a lot... i love it...

[@WiseOwlTutorials]

You're welcome, thanks for watching!

[@novicerider751]

I've noticed that .AsTable.ToExcel is both simpler and quicker compared to processing the table manually. However, when used within a loop, is it possible to scrape the data onto a single worksheet instead of adding to a new one on each loop?

[@WiseOwlTutorials]

I agree about AsTable.ToExcel! And yes, we can certainly build a list in a single worksheet, you simply need to find the current bottom of the list for each iteration of the loop. We have a video which goes into some detail on finding the end of a list which you can see here kzbin.info/www/bejne/pnqVoatogNSUprc One reasonably reliable way to do this is like so: Private cd As Selenium.ChromeDriver Sub BuildListOfTables() Dim tbls As Selenium.WebElements Dim t As Selenium.WebElement Dim OutputSheet As Worksheet Set cd = New Selenium.ChromeDriver cd.Start cd.Get "any URL with some tables" Set tbls = cd.FindElementsByCss("table") Set OutputSheet = ThisWorkbook.Worksheets.Add For Each t In tbls t.AsTable.ToExcel target:=OutputSheet.Cells( _ RowIndex:=OutputSheet.Rows.Count, _ ColumnIndex:=1).End(xlUp).Offset(1, 0) Next t End Sub Hope it helps!

[@novicerider751]

@@WiseOwlTutorials You're a life saver! Watched the video too. Thank you :)

[@WiseOwlTutorials]

@@novicerider751 You're welcome! Just added a reply to your earlier question about the NextURL not working on this video kzbin.info/www/bejne/n5qUfaucesuMapY

[@stevethorne44]

I have really found this useful to learning VBA. Thank you for the helpful video and explanation. I wonder if anyone could help with the following. I am following along and writing the code. at 53mins in the video where I run the code in my work book. This grabs the table and populates it into the Workbook, but starting on Row 5, Rows 1-4 are blank. It just starts with the data, and does not even get the headings. Any advice?

[@WiseOwlTutorials]

Hi Steve! I'm not quite sure why you're experiencing different results. Have you tried downloading the completed version of the workbook to check if the same thing happens? You can find the link in the video description.

[@stevethorne44]

@@WiseOwlTutorials hi wise owl. I meant to comment back to my original post. I put a 3 second delay into the code, to allow the page to fully load at that point. That seemed to solve the issue. It is now capturing all the results. With little VBA knowledge I managed to resolve it. I've obviously picked up that knowledge from your video. Thank you very much.

[@WiseOwlTutorials]

@@stevethorne44 Good stuff, happy to hear that you got it working!

[@rahulshewale9206]

Thank you sir

[@WiseOwlTutorials]

You're welcome Rahul, thanks for watching!

[@samuils]

All is great but for me who is trying to print out a single element, I might have missed it, and see all built on extended functions instead. Have to search all over the place for a simple version.

[@especuladorinteligente9428]

My code which scrape using IE stopped working, it says The site you are trying to acess doesn't work with Internet Explorer, (then redirects me to microsoft edge) So i think u need selenium right? However, is there any way of just chaning one line of the code? amazing leasson, however im tired of trying to learn haha

[@WiseOwlTutorials]

Yes, that's right, Selenium is what you need if you want to control other web browsers in VBA. Sadly the object library for Selenium is quite different to the Internet Controls library but the good news is that I think you'll find that it makes a lot of things easier. Good luck!

[@kjesu2715]

Thank you...

[@WiseOwlTutorials]

You're welcome, thanks for watching!

[@excelworld2410]

Your video is really awesome. Is it possible to scrape data from amazon, ebay, alibaba, aliexpress? I have tried but am unable to scrape using selenium VBA. If possible then please create a video on this. Thanks a lot.

[@WiseOwlTutorials]

Yes you can, there's a bit on Amazon in this video kzbin.info/www/bejne/Y5zNopJunp6Vq68 The main problem you'll have is keeping your code up to date when the website inevitably changes!

[@kabukijoe99]

is there any documentation on writing code for vba + Selenium? thank you

[@WiseOwlTutorials]

Not as far as I know. SeleniumBasic is an independent project, VBA isn't an officially supported language for Selenium.

[@SamehRSameh]

Can we import data from application ????

[@Machiuka]

Awesome tutorial. It gives me an error "td" element not found. I didn't figured out, and finally I've copied your code. Thank you!!!

[@WiseOwlTutorials]

You're very welcome, I'm happy that it helped you!

[@TheFabsanchez]

Hello. How can I do if the website from which I need the information requests login before accessing the data? Thank you so much

[@WiseOwlTutorials]

Hi Fabian! It depends on the website you're using. If there's a convenient form on the page to enter your Username and Password you may be able to use Sendkeys to type your credentials and then Submit the form. If the site uses Windows security you might find this video useful kzbin.info/www/bejne/hanbZ6dsd6p9d9k (sorry it uses Internet Explorer!). If the site is more complicated than that you may find that you need to use a provided API and get involved with OAuth2 but I don't know enough about that to give you any good advice, sorry! stackoverflow.com/questions/55882439/vba-api-get-that-requires-request-token-authorizing-and-access-tokens

[@marcq877]

Very interesting chrome driver for VBA. How can I find a specific value inside a child "xxxx" where I know the parent tag TEXT I NEED Without it going all over the page tags. Thank you

[@WiseOwlTutorials]

Hi Marc! I'd check out part 57.4 which covers lots of techniques for referencing elements. I'd focus on CSS selectors or XPath to get what you need. I hope it helps!

[@peterwaynealjibe2165]

Hi Andrew, thanks very much for this tutorial. would like to ask, will there be other video tutorials on how to do web scraping to those sites that require password using selenium and vba?

[@WiseOwlTutorials]

Hi Peter! I don't have anything planned on this topic, was there a specific website that you were interested in? In simple cases you can automate logging in using the same techniques for entering data in basic web forms (as we did in this video). Of course, not every website behaves in the same way!

[@peterwaynealjibe2165]

@@WiseOwlTutorials thanks very much for the hint!

[@WiseOwlTutorials]

@@peterwaynealjibe2165 No problem, I hope it helped!

[@leebrook2874]

Using Selenium how do you return text from a Class Element into Excel row A1