Scraping Dark Web Sites with Python

jh.live/flare || You can track down cybercrime and manage threat intelligence or your own exposed attack surface with Flare! Try a free trial and see what info is out there: jh.live/flare
🗨️ Scraping onion sites across Tor hidden services in an automated way -- quick crash course on some of the command-line tools or Python libraries that can help you create your own custom monitoring tools for cybercrime. 💬
Learn Cybersecurity - Name Your Price Training with John Hammond: nameyourpricetraining.com
📧JOIN MY NEWSLETTER ➡ jh.live/email
🙏SUPPORT THE CHANNEL ➡ jh.live/patreon
🤝 SPONSOR THE CHANNEL ➡ jh.live/sponsor
🌎FOLLOW ME EVERYWHERE ➡ jh.live/twitter ↔ jh.live/linkedin ↔ jh.live/discord ↔ jh.live/instagram ↔ jh.live/tiktok
💥 SEND ME MALWARE ➡ jh.live/malware
🔥KZbin ALGORITHM ➡ Like, Comment, & Subscribe!

[@MarkParkTech]

You shouldn't recommend torpy since it only supports v2 addresses... which haven't been supported on tor since July of 2021, and they haven't had a new commit in 3 years. Essentially, torpy will not or should not work with tor anymore and until they start updating the project ( unlikely ) or someone forks it, it shouldn't be promoted.

[@samswag7243]

Damn 2021 was 3 years ago

[@DouweHummeling]

Here, have a like for the hotelroom video.

[@parsaz3277]

Dislike button ⬇️

[@home1250]

By far my favorite channel for authentically practical guidance Thank you .

[@xp__d4600]

Respect ngl, this man took his full setup to a hotel across a long trip just to make this video available for us.

[@kyleheaney3734]

His full setup equalling a laptop and microphone?

[@xp__d4600]

@@kyleheaney3734 laptop, mouse, microphone, camera

[@DoingFedTime]

(Here from linkedin btw) This is badass. Great work on the video, seriously. Definitely work that subscribe (I already was), and a thumbs up!!!! 👍👍

[@user-bc4gp4nr5v]

collab w bro

[@DoingFedTime]

I'd love to. Dudes a legend. @@user-bc4gp4nr5v

[@Indra-bc9xn]

@@user-bc4gp4nr5v dp

[@mogulgrp]

This is an amazing video, and the hotel room was the perfect fit for a darkweb video. 🙌🙌

[@GeorgeAlexanderTrebek]

More videos like this! This is extremely useful and really applicable to many things. Love basic concepts that can be applied to many things.

[@tetttettamilli6761]

@JH - Excellent information 👏👏👏. I can use this Python "web scraping" utility right now.

[@mrgammaray343448]

Absolutely love this. Really am grateful for helping me expand the knowledge and projects ideas 🙏🏼

[@SplitUnknown]

Looks like john is recording somewhere else. Thank you for the video 😄

[@AB-nv6fh]

Amazing vid u could use libraries aswell like selenium or beautifulsoap

[@axelbruv]

Love your videos.

[@smwx4]

Interesting, I’ll watch later🎉

[@LateralNW]

Great info thanks.

[@rogerioabreu3081]

Great Work John! When are you coming to Germany?

[@dadamnmayne]

sponsor transition smooth, on point and relevant

[@neutrino2211_]

Had this on my second screen. There was a Sponsor?!

[@dadamnmayne]

@@neutrino2211_ mans becoming a smooth operator

[@Crysal]

All you ever need it a for loop and curl

[@David-ce1ux]

Would Microsoft Defender for Endpoint do the same job as Flare or better?

[@user-nr1qk6oi7g]

Hello. Could this technically be used as a free proxy?

[@khanabdulmuhammad5625]

00:04 Automating web scraping of dark web sites using Tor 02:15 Enabling control port and authentication methods for security. 04:24 Monitoring dark web for security threats and data exposure 06:32 Scraping dark web sites can provide valuable information for businesses and individuals. 08:35 Using Python to scrape dark web sites 10:47 Accessing dark web sites using Python Requests library. 13:00 Exploring a ransomware gang's exit scam and law enforcement operation on a dark web site. 14:57 Scraping Dark Web Sites with Python 16:52 Configuring control Port for authentication and IP manipulation 18:37 Explore dark web marketplaces for insights.

[@KwiatekMiki]

Did you use some sort of website or an AI to write this? It's weird that you did it in around 3 minutes for a 19:28 minute video.

[@ilovebobrov88]

​@@KwiatekMikiObviously.

[@khanabdulmuhammad5625]

@@KwiatekMiki speed matter

[@t-ree]

AI

[@redarrowgaming4700]

My issues are with finding all the different onion links that could be used to scrape, tor links themselves are hard to find, is there anyway of automatically crawling for onion links to scrape?

[@KwiatekMiki]

Here after the video and it was good.

[@makouru9598]

would it be possible to write a scraper, where you feed it with an adress and he is searching for adresses in this adress and stores them in a data base besides a bunch of information. At the beginning you manually feed it with links and later it is just constandly running and storing links in its database... and here we go.. torch 2.0? Is it more or less working like this? A search engine?

[@user-ef1rs5to5y]

This dude is a genius.

[@ravetime8394]

How would this work when darkmarkets require captcha to view there site? Thank you in advance

[@sorrefly]

How do I know the python library is not injected with malicious code?

[@gjkrisa]

i’d like to have a tor crawler to index all the sites and their pages. then that could lead to the rest

[@successtv360]

Hi. I want to extract the data (result history) on a virtual gaming site. can you help me?

[@user-bs8mj4hu2u]

Can you explain activating ssh with proxy on the victim’s device to configure the firewall?

[@jmr]

Sadly this isn't practical for the dark sites I want to read. They get hammered and have too many anti bot protections. I'd love to keep up in an RSS reader or something.

[@VaibhavShewale]

at first i was confused on how to scrape the or site but after watching it i got it

[@RandomGeometryDashStuff]

11:18 why RequestsTor needs cport=9051 but there was no 9051 in curl command (08:40)?

[@oriyadid]

As he explained at 8:40 curl does not care about that port, it is not where the proxy resides. cports stands for control ports, and that's what 9051 is for. Curl only needs the proxy, which is on port 9050, and at 11:18 he indeed did give RequestsTor port 9050 as the proxy port. Edit: as for why RequestsTor even needs the control port, no clue

[@user-be2bs1hy8e]

not going to lie - i relate better with legacy setup pf john hammond, +1

[@Billblackvip]

Man sad, man see video, man happy

[@HeartsOfMold]

Man gay

[@Billblackvip]

😂😂😂

[@CyberSecForce]

GREAT WORK

[@wahakadd5181]

Enjoy your trip and don’t forget to visit Morocco you going to like it 👍👍👍👍 🇲🇦🇲🇦🇲🇦

[@ReneSanchez6502]

bro i got like 30 ads in this video what's going on

[@NeroCat9999vr]

That’s called “KZbin.”

[@user-rz3yt4rt2y]

I got zero its called "ad blocker"

[@NeroCat9999vr]

@@TheRealLoganJosh No. They *can* select when. I didn’t get that many ads in this video which proves the creator made zero input on when, where, and how many ads would play on this video. It was KZbin doing typical KZbin shit.

[@seethekek4647]

Bro really making a tutorial on how to make a dark web crawler to find illegal websites

[@ambroserapose5082]

Hey John, I am a victim of someone hacking my multiple accounts gmail microsoft Facebook twitter etc maybe through my phone or somehow they got access to my Google password manager, Is there any safety steps I can take other than changing password and adding 2 factor authenticator app? Any help is appreciated.

[@paralies1276]

Use a password manager like keepassxc to store passwords and generate random ones, don’t re use passwords

[@digitalcivilulydighed]

Totally not hateing u!

[@loudsquad2324]

In a hotel, might as well do some darkweb work ;) lmao

[@mournfulpup]

you have way too many ads. This aint a vevo music video lil bro.

[@avs_smart_systems]

Sorry for a millisecond though of Seth Rogen

[@uuu12343]

Hotel room videos feel oddly relatable...

[@DccToon]

were you sponsored by flare or did you like flare

[@xuxusito]

Of course he were sponsored.

[@bettergamingseat3171]

Scraping mhm

[@conan1231]

The "Torpy" Project seems pretty dead to me. Sad.

[@j00500hall]

Interesting video. Great advertisement Segway. Incredibly useful info, dreading what you could actually do with this.

[@hgvhjfcjdudrsxhxj]

nice new darkweb video

[@Tf2_Update]

Blud is on the run

[@professional.hacker.]

You set it to zero so the police won't show up at your door right?

[@sgotach_7581]

I immediately know people gonna use this for ummm Effed up means 🤢💀

[@hedikintheoriginal]

upvote for a bathroom video next :D

[@unknownlordd]

John notice me before I leak your public IP 🥺

[@hamedranaee5641]

Oh I missed the first like :( , anyway I love you John :D

[@Ruhgtfo]

Pm tranquilizer

[@shakibbro2]

please 1 video how to hacked gmail password please please new video 🙏🙏🙏🙏🙏🙏

[@hedgehogform]

Say bye bye to that hotels wifi network.

[@us_f4rmer]

Nice one KZbin, changing the bell to either ALL or PERSONALIZED automatically defaults back to NONE. At least with Edge, maybe try a different browser though.

[@tojo1598]

Websocket ko bina authentication ke kaise implement kare? Mujhi 403 error de rha hai maise /ws ko permitAll kiya tha

[@warpathpro]

Super duper 420 eyes 😂

[@dilshanperera9694]

damn

[@RussianPutin506]

Dm

[@jbmarkowicz3328]

LOL you are wild showing people this. You know what's within that. Now you're going to get some little kid like "Let's go get the bad guys" and accidentally come a cross a site which will scar them for life. Well done. HAHA.

[@darkheart8961]

This is soo old school

[@user-td4pf6rr2t]

so not trying to be buzzkill but mine didnt work

[@bettergamingseat3171]

Don’t use flare it will leak more info from you and gather it and pin it to you and you don’t want that don’t fall for this use your own infosec tools

[@LuvVortex]

I really don't like y'all nsa ...... I will be getting biofuels release record

[@AspergersStudio]

R u on crack

[@user-yw1lb5ru7p]

Old market of edg ppl y can by weapons womens any thing y can build empire with it...😂😂like the real map of world y know map lead to treasure sow they give the wrong map😂😂😂evwn earth map

[@KellyMahlangu]

Hello, I want to get into the the software engineering but because of financial sitiations I am having issues paying for tuition So if there is anyone who is willing to help by offering to pay for small tasks so I can afford to pay monthly payments for a student loan please let me know

[@kalp_it]

use vim instead of nano if you have balls.

[@bfx8185]

Please stop it! Advertising on you channel is overwhelming and even though I was considering being a member I considering to put you channel on my black list in the end.

[@toinfinityandyourmom2219]

And now millions of script kiddies are going to waste tor bandwidth... nice.

[@razdingz]

i like woman_SytsOnMyFayce