How Traceroute Works (Building a Movie Scene 'Trace' Map) - Computerphile

[@harry.tallbelt6707]

Ahah, I'm starting to get a feeling that almost everything about the modern internet is a hack on top of things that weren't designed for this. Great video, thank you!

[@durnsidh6483]

Why do you think internet security is so bonkers?

[@Vampyr787]

Of course it is. The internet itself is hack on top of other networks that were there before like phone network and cable tv networks. The biggest problem is always backwards compability. You can't just introduce new things and force everyone to throw away their current network devices to buy new ones so everything has to be backwards compatible.

[@BattousaiHBr]

im a net admin at an isp and i guarantee you that is the case. bgp is notorious for having no security whatsoever and depending on trust to work as expected, with other hacks like rpki being pushed onto the community with varying levels of success.

[@myaccount99000]

Still waiting for the "modern internet".... Seems to me largely identical to what existed 50+ years ago lol

[@LabGecko]

@@myaccount99000 Honestly the "modern internet" is made up of people collecting on various apps on their phones. I wish they'd make a new net with end-to-end encryption but that costs money no one wants to pay and would cut out data a lot of corporations make money on.

[@jjdawg9918]

So happy he didn't stop with a simple explanation of the mechanism. All the caveats,hacks and misuses were so beautifully explained.

[@rationalraven8956]

I've been using traceroute for 20 years and never actually knew how it worked, fascinating!

[@charlesenfield2192]

Traceroute can take a little practice to use effectively, to help shorten the learning curve I will add that there are three common reasons why we don't get TTL exceeded messages from routers. 1) As mentioned in the video, some operators think sending ICMP messages to external networks is a security risk, so their routers have filters that only send these messages to other routers within their network. This thinking is mostly outdated, at least as it applies to networks meant to host publicly-accessible content, and it's not the main reason we don't get responses. 2) ICMP messages must be sourced from a specific IP address. Some IP addresses are internet routable (AKA, public) and others are not (AKA, private). Routers directly connected to "the internet" (AKA, border routers) usually source these messages from a public IP address, whereas routers within a private network usually source ICMP from a private address. Routers using a private address usually send the TTL Exceeded message, but it get's dropped at the border because of the private source address. This is the most common reason you never hear back from certain routers during a traceroute. 3) Routers can get very busy. Router software places a lower priority on sending ICMP than on routing packets, updating routing tables, etc.. Busy routers are a common reason why you sometimes get a response from a certain router and other times you don't. If you use traceroute to assess packet loss along a route, don't assume that missed TTL Exceeded messages are the result of packet loss.

[@iabervon]

I believe your packets may be going over a segment where routers aren't speaking IPv4 and are encapsulating your ping in something else along with copying the TTL back and forth. Works fine if the packet makes it, but if the TTL expires, they may not have a response they can give.

[@rsilva2k5]

Also, some routers will only delay/drop packets when they are actually busy, while others will have explicit firewall rules to rate ICMP requests/responses towards itself, which means these routers will delay/drop ICMP replies even when they are not busy.

[@lawyupang]

FYI, by default, linux traceroute is using UDP instead of ICMP. Also, router will respond with preferred IP address of the interface that receives the packet.

[@charlesenfield2192]

@@lawyupang It doesn't matter what type of IP packet the host running traceroute sends. All that matters is the TTL. When the TTL is exceeded the router will either do nothing or send an ICMP packet. UDP is not an option.

[@cda32]

yes indeedy, some peering links and MPLS nets will route via IP blocks which are not advertised in the global routing table. They may not use IP at all.

[@TheStabbyCyclist]

Currently covering ICMP in my TCP/IP class and I'm taking a Python class as well. This video could not be more relevant! Love it.

[@SpiderTalon]

I would like to see the graphics updated with each hop being a different color. It would be clearer when there is a hop. :-)

[@klaxoncow]

Also, to be like the movies, it should have shown the world map straight away and then drawn each line one-by-one. Like, show the world map, run "traceroute" in the background (redirect stdin and stdout) then parse out the IP address as it comes in, drawing the great line and then you've got an animation, like in the movies, where you see the route drawn out hop-by-hop, like you're "tracing the call". This is Hollywood. Presentation is important.

[@monstertrucks9357]

@@klaxoncow Don't forget it needs to be mostly green on black. Preferably with some ones and zeros cascading down across the screen somewhere

[@willbradley1734]

Or use dots.

[@ShooterSF]

@@monstertrucks9357 It'll also need a beeping noise that gets faster as the route nears the final destination

[@mikejohnstonbob935]

when multi-planetary internet is a thing, some geoip responses will return the coordinate for the center of the earth. imagine how annoyed the magma monster will be when humans pester them about getting hacked

[@arik_dev]

I'm going to make a killing on my Mars CDN startup XD

[@esquilax5563]

Haha! But it's hard to do proper hacking when each packet takes several minutes to reach its destination

[@Tiddle_s]

"Makes its way down to Sydney for the final bit" That's Melbourne. Sydney is roughly where the line comes into Australia. Maybe the drawing is a little messed up, or it took a hop to Melbourne (maybe to join aarnet?) before coming back up to Sydney

[@richardclegg8027]

Oh gosh -- how embarrassing. I hope my cousins in Australia don't see that. Thanks for the correction. You could be right that it has to backtrack a little to join a different network. (Australia is a little "distorted" in Internet delay maps because it doesn't have many costal entry points.)

[@1TW1-m5i]

Unless they reached a server sydney uni has in melbourne for some reason?

[@Tiddle_s]

@@1TW1-m5i more likely it's just guessed the location incorrectly (as with the US hops) but fair point.

[@SeanBZA]

@@richardclegg8027 Cape town your route would, depending on the actual interconnect your ISP uses, actually have been around the western coast on a long cable. Then a few hops in South Africa, and off to the UCT servers. My geolocation always comes up as 600km away, in Rivonia Gauteng, and i see yours does as well, so you are meeting an Akamai CDN, or the servers themselves are actually located there, and not at UCT. Big server farm there, who have as selling point that they are very prepared for DR, seeing as they still have power when Eskom does not, and also have a month or more of fuel at any time for the systems to run.

[@silkworm6861]

Haha, came here to say that. Great video though!

[@raunakmitra7868]

SHEER CO-INCIDENCE! I had begun studying the concepts of IP Packets and network troubleshooting using CMD lately. Overwhelmed to find this top notch video explaining the tracert command in Command Prompt. Thank you so much

[@satepestage3599]

More from this guy please, he's very good at explaining things.

[@u0000-u2x]

an interesting thing to do is to take a closer look at the nodes before and after the oceanic crossings and correlating them (by taking their names and IPs) with underwater fiber cables. you can find out exactly the physical route your packets took, which company owns the cables, when they were installed and other cool stuff :)

[@LupusZockt]

Best explanation I've ever heard about traceroute. Thank you! This is great!

[@binshumeshsachan2051]

I have just finished reading this topic. And the video arrives. What a coincidence.

[@m1geo]

Whey! It's Richard. I worked with Dr. Richard Clegg during my PhD at University College London. Now at QMUL. 👌

[@jonathanjay2594]

This guy needs to make more videos, he's a good teacher

[@964tractorboy]

One of my favourite Computerphiles ever. Great stuff. I'm also a big fan of QMC. Richard's haircut may have inadvertently put him in the frame for the next Bond villain.

[@subtledemisefox]

I was thinking more David Lynch

[@percyblakeney3743]

Almost all of those who participate in these Computerphile videos provide little to no background noise or "stuff to look at." Which not only serves as a security measure on their behalf but also as a way for people like me to become less distracted while trying to learn.

[@Deckardcainhasnomana]

Am I the only one who sees an older version of the Technology Connections guy?

[@rolfs2165]

Now that you mention it …

[@chaparmusic]

Whoa...

[@U014B]

I was thinking more David Lynch, but yeah I can see it.

[@Diapolo10]

Yes!

[@paulmichaelfreedman8334]

I thought the one in this video looks a lot like Ronnie Barker in his late 40's.

[@FlyingThunder]

even if i already knew all of this, this was really interesting to watch, very well done video

[@zyansheep]

The person who lives in the middle of America should put up a sign outside their house that says: IP GEOLOGATION IS NOT VERY ACCURAT. and make it large enough to show up on Google maps

[@salerio61]

The poor sods have had armed raids, angry people turning up, drugs raids, everything. They are well fed up with it.

[@HECKproductions]

to make middle of america people look illiterate?

[@G12GilbertProduction]

By what geographical or topological argument you @Zyansheep say you doesn't may accurative IP geolocating?

[@salerio61]

@@G12GilbertProduction I thought the 'Spoons were closed for the lockdown

[@yolamontalvan9502]

It would have to be in Spanish because the middle of America is Panamá, that’s why it’s called Central America. I’ve crossed the Panamá Canal at night and it’s pretty beautiful.

[@wp5355]

Excellent explanation - first time I have heard from this person - hope to hear a lot more from him.

[@AndreAmorim-AA]

That classic 2D map scene really reminds me one of my favorite game ever .. uplink from introversion software ..

[@richardclegg8027]

Another reason I wanted to create the graphic output. I liked that game too.

[@Henrix1998]

I have tried it but never really have understood their games

[@thedragonrises6882]

I was really digging into the story but never got that chance to finish it. Fun fact about Uplink: there was very little to no publicity done for the game. It was all done through word of mouth.

[@gloverelaxis]

it sadly runs really poorly on modern computers. deserves a remaster that takes into account varying resolutions!

[@AndreAmorim-AA]

@@thedragonrises6882 for me the game was funny because at that time introversion was really small company.. they used to sell binaries and the source code in c/c++ and reading the code is like reading gnupg source code .. functions with descriptive names like "bumpkeys" you think.. did that function recovers a lost key?🤣

[@DenilsonSa]

"xtraceroute" is (or was) a more advanced version of this Python script: it displays the hops on a 3D globe. The project seems (sadly) abandoned, though.

[@RonJohn63]

Regarding the paths going through the US: there's almost certainly *more and higher bandwidth* cables running across the Atlantic, across the US and then across the Pacific than across Russia & China or Egypt/Suez & the Indian Ocean.

[@G12GilbertProduction]

Or Black Sea beneath the Lesbos and Rodos islands? Or Kuweit? :P

[@nahco3994]

Probably right about Russia. I'd suspect that the biggest cities (15 cities with pop > 1 M) probably have decent connections between them. But the problem is that the eastern most cities in Russia are actually fairly small (even Vladivostok only has 600 k), and the Far East region in general is rather weak economically and in terms of infrastructure when compared to the west of the country. Also, how much traffic is actually routed **through** China anyway?

[@RonJohn63]

@@nahco3994 who (besides the PRC) *wants* to route data through China?

[@harriehausenman8623]

What a massively cool dude! Thanks for the vid.

[@Cyb3riano]

Very good explanation and nicely rendered. I watched paying much attention while on the bus. Regards from Argentina 🇦🇷

[@ricardofynn]

Love dot-matrix green papers used in all videos to explain everything.

[@ElectricEvan]

The Tor project used to have a GUI called Vidalia that showed the tor nodes on a map that your traffic was using.

[@riverlance9647]

10:18 , that a coders whole life

[@mith873]

i was looking for this for quite a while and i see this in my recommended today thanks a lot

[@rsilva2k5]

One thing that really bugs me when people talk about traceroute is that they never bring up the fact that it does not show you the full picture of the internet routing of your packet, which leads to a lot of misconceptions and misinterpretations about what's going on. You see, packet routing on the internet is inherently asymmetrical, meaning that the path taken from A to B might differ from the path taken from B to A. The chances of A->B being different from B->A increases with the number of hops on the internet. When you traceroute from your machine to a webserver like in the video, it only shows you the A->B path. If you want to know the B->A path, you MUST traceroute from the webserver to the machine, otherwise you'll never know. This is important because sometimes you might get false-positives when, let's say, you're trying to identify which hop on the internet is messing with your packets. There's one such example in the video: when he gets * * * from some hops, one might think that those hops are the ones filtering ICMP packets, because you never see anything beyond them. In reality, those hops might be answering ICMP, but you never get to see their responses because the return path is different, going through another router that actually filters ICMP. Your only chance to pinpoint which router is filtering packets in this scenario is to have a traceroute from the destination towards yourself, and then comparing both traceroutes.

[@fakename8749]

Reminds me of the nights I've spent playing Uplink: Hacker Elite back in 2000s.

[@Scimu]

It's on steam if you ever want to reminisce :)

[@tactikool4740]

I have spent years in this industry and trying to perfect my cyber skills and no one has explained this topic to me so well.

[@diracflux]

@14:00 I’m no geographermancer, but I believe that southern point is Melbourne, Victoria.

[@quiversky4292]

That traceroute world trace actually landed in Sydney on the East Coast, then went down to Melbourne

[@ARUCARDFTEPES]

I don't know if I actually learned anything, but that was really interesting.

[@michaelsmitasin4301]

Speaking of traceroute, hello from Lawrence Berkeley National Laboratory!

[@myce-liam]

Would love to see more from Dr Clegg

[@gargamelgamingotherstuff6727]

AS a computer nerd, i finally understand the asterix results, i never understood this and never googled this result. And i grew up with commodore 64 era. :D

[@ShakalDraconis]

This sent me down a mental rabbit hole of "what is the center of the USA... say we define that as the point that has minimum collective distance to all other points, yeah probably someplace in Kansas or Missouri or thereabouts... but wait, Alaska and especially Hawaii are going to pull that way off to the north and west... and what about territories, what else do we count..."

[@patriark]

I'm a little giddy to see the Steam icon on his desktop. Even very smart people need leisure

[@gregf9160]

You had me at Traceroute and Python 👍

[@H76Pro]

Excellent explanation!!

[@HebaruSan]

That is an impressive number of GNOME Shell extensions in the top bar

[@MagicPlants]

Don't be so nervous, you're doing great! :)

[@bartz0rt928]

There was an article on the news a few days ago about the guy who lives in the middle of the Netherlands, he has that same problem. He even runs an IT company from there, which makes him a more "plausible" suspect.

[@et0san]

Hey, that was actually really nice. Thanks for sharing :)

[@Bames007]

Can we get the python code?

[@dandan7884]

there should be a github account for this channel with all the code available in the videos. preferably with a Dockerfile or docker image configured to plug and play

[@Computerphile]

Just added github link in description HTH -Sean

[@nonasuomynona1734]

@@Computerphile Thank you

[@richardclegg8027]

Warning -- it is not production quality code. :-)

[@hashiromer7668]

@@Computerphile I still can't view it

[@danielyoung_]

Really interesting video!

[@lawrencedoliveiro9104]

0:14 Another world map without 🇳🇿 ... sigh ... 0:37 Mentioning that he wants to ping Auckland just doubles the irony ... 0:44 ... OK, that’s a better map!

[@michalnemecek3575]

11:10 is this why sometimes I log in from Prague, despite being in Jindřichův Hradec (about 100km southeast)?

[@BobChege]

Great explanation!!!

[@surferbum618]

Awesome video

[@jhonbus]

It's a lot cheaper and easier to put internet cables under the sea than it is to put them across land. If you go overland it's also a lot easier for someone to find and destroy the cable. Even the eastbound routes go via the Mediterranean and via Suez to the Red sea, Indian ocean. Imagine the nightmare of arranging to have super important global infrastructure go through the dozens of countries in the Middle East, with everybody wanting a cut and threatening to cut it off every other day.

[@ariaghafari73]

On line 74 at 12:42, there is a comma after the word line. Does anyone know why? As far as I can tell, it does nothing but I don't have much experience with python.

[@richardclegg8027]

The mdrawgreatcircle routine returns a list with one object. In python this construction will take the first item in that list. Actually, I don't need the return value so it could be deleted I think (not tested that).

[@LegendBegins]

Good 'ol Tracer T. Ultimate hacking tool.

[@weksauce]

Geo center of America is unpopulated. Geo center of either South Africa or a specific city was for a while a dude's house. Interesting interviews on KZbin about the phenomenon described at 12:00 that don't and haven't happen(ed) for geo center of US!

[@gordonrichardson2972]

14:12 Cape Town is on the coast, not in the middle of South Africa. The undersea fibres are not over the land.

[@richardclegg8027]

Yes, I've visited Capetown and I can confirm it's definitely on the coast. The geolocation only shows us where servers are -- none are underwater (well not usually), so we will never ever see "hops" in midwater. As I explain, sometimes the IP geolocation only gets country level accuracy. So if the service I used only knows "It is in South Africa" rather than Capetown specifically then the end point is just the centre of SA.

[@ankittayal8291]

Well explained 🔥

[@minghueileong]

Reminds me if McAfee security center / firewall that used to have this traceroute with map feature for tracing attacks.

[@tensevo]

Actually brilliant.

[@AndreAmorim-AA]

Quick question about Geo tag, Geo fence, etc.. is Geohash a good way to work as unique key for a dataset? I mean it is strong enough to a hash collision etc..

[@Cypeq]

The asterisk more often then not are the huge data centres converging major internet traffic. Ping is rejected there for load reasons you simply don't want any unnecessary overhead. Security reasons are obvious.

[@lawrencedoliveiro9104]

7:28 Look at all the 10.* addresses up to step 7. That’s a lot of ISP-internal routing going on!

[@CautionCU]

Nice explanation

[@vonyp6018]

You have also forgotten to mention that the hop location depends on where the IP address was registered. The test is a good general illustration of how the internet works but the traceroute path is is totally inaccurate.

[@richardclegg8027]

I wouldn't say "totally" inaccurate but certainly it can be really misleading. But yes, I do see servers I know are in China geolocated in Europe. It's a limited tool but of considerable interest. There are other techniques you can use based on triangulating delays but you're correct that there's no perfect way to turn IP into lat-long.

[@kamausamuel9172]

Very Insightful

[@macfhlannchadharonan4668]

Is there not an ICMP trace route functions that is used instead of the time exceeded ICMP message?

[@du42bz]

No

[@RatnamalaT]

Sir thanks for clear explanation....I want the videos how wires harkand putty tool works please keep a video on that.....

[@jms019]

What about the box on Big Ben ? The museum at Porthcurno is superb for anyone vaguely into communications

[@ncot_tech]

a video on why the first 7 hops shown at 8:26 are all private IP ranges could be interesting. You're on a mobile broadband connection, aren't you?

[@FerrybigGaming]

Note that i the modern age, we use IPv6 more frequently. TTL (time to live) has been renamed to hl (Hop Limit)

[@tonydarre9954]

Does pathping have the full functionality of both ping and traceroute?

[@lawrencedoliveiro9104]

12:33 Best to avoid spawning a shell in your subprocess if you can avoid it: spawn the traceroute program directly.

[@rujotheone]

Why?

[@lawrencedoliveiro9104]

1) Overhead. 2) Special interpretation of shell characters introducing unwanted effects.

[@paulminshall8793]

I seem to remember that there is a location in the US which continually gets law enforcement rocking up to their door, since it is used as a default location for tracking apps.

[@Omnifarious0]

The geolocation people should maybe randomize the latitude and longitude to be somewhere within the geographical area generally indicated by the IP address.

[@kamilkardel2792]

I've tried looking up that longitude and lattitude in the USA (latitude":37.751,"longitude":-97.822) and found it to be somewhere in the middle of Cheney Reservoir, located west of Wichita, Kansas.

[@richardclegg8027]

If you google "How an internet mapping glitch turned a random Kansas farm into a digital hell" you should be able to see what I'm talking about. (Seems like the link gets removed when I include it.)

[@theRealtensigh]

Is the code available somewhere? It would be a great teaching tool. Of course, so is the video, but that code would be fun to work with. EDIT: Never mind, saw the link in the comments. Damn that "SHOW MORE" - oh wait, maybe Praise that "SHOW MORE"...

[@krissp8712]

Maps without New Zealand on them! (the green one, not the UoA one)

[@IanTester]

I remember back in the day running a traceroute from here in east-coast Australia to... that MP3 website in Russia of dubious legality. I can't remember the name and I guess they're long gone now. Anyway, the route went across the pacific, across the U.S., across the Atlantic into Amsterdam, A few stops through Europe, and into Moscow. Not the quickest route because the internet was and is still very U.S.-centric.

[@РусланЗаурбеков-з6е]

Don't even ask, why you was so interested in pirated MP3s store in Russia... )))

[@LabGecko]

@@РусланЗаурбеков-з6е fewer / faster hops makes those MP3s download faster, of course XD

[@darrentravi6069]

This video is great I am however really sick of the adverts

[@laykefindley6604]

That last point is very interesting. If we start to have more solar based power, I wonder if the machines will intelligently reroute traffic through routes that are in maximal solar and/or wind green energy while avoiding paths on the shadow side of the planet? Thoughts anyone?

[@LabGecko]

Not without someone cataloguing which systems are "green" and which aren't, and then someone coding internet protocols to favor those green systems. Side note - as soon as such a system is put in place expect hackers to spoof their systems to look like green computers to get priority, or spoof them to non-green to more easily avoid detection

[@paulmichaelfreedman8334]

My goodness, I thought for a moment he was Ronnie Barker in his 40s. Incredible likeness.

[@smurfyday]

Why does the computer take so long to determine if password is correct at login? Windows and Linux, at least, even when the password is literally blank, takes awhile to come back rejected.

[@KC-nd7nt]

Rooter? Like the drain cleaner ?

[@userou-ig1ze]

no captions and autocaptions disabled, sad! Can't watch with audio and no headphones nearby

[@benw-l7k]

watch it later then? the video isnt going anywhere

[@userou-ig1ze]

@@benw-l7k ... sure there are many solutions to a problem. Some involve exactly toggling one button, others require more

[@Minefortress21]

Hello, Can you guys make a video on pagination? I believe it is an interesting topic and I haven't been able to find relevant videos on your channel. I would be great if you consider that :) Thankyou

[@juicebox86]

We now use Akamai at work for our DNS. Using DigWeb interface you see Akamai and not our servers.

[@twistedddx]

Doing great right up to the point he points on the map at Melbourne and calls it Sydney.

[@richardclegg8027]

Sorry about that. Geography of Australia is not my strong point I'm afraid.

[@dandan7884]

omg i also use geany everyday to code xD. i choose simplicity over aesthetics every time when on linux use geany when on windows use notepad++ best editors ever for me

[@richardclegg8027]

Yes -- it does everything I need without getting in the way.

[@boaro6604]

I use it on windows as well. It has all the necessary functionality without being bloated with features. Always love seeing geany users in the wild.

[@colingreysful]

This seems to be written to work on Linux. Could it be adapted for Windows?

[@Tigerzplace]

Install python and its requirements, I hope it will work on windows as well.

[@FriedEgg101]

When I use my phone as a hotspot (EE network), websites think I'm in Croydon for some reason. I'm not in Croydon. But when I use my home broadband websites know exactly where I am, down to about half a mile, which is a bit creepy.

[@richardclegg8027]

It all depends where the first place on the network with an IP address is. (I'm also using an EE hotspot in that video.)

[@BattousaiHBr]

surprised he didnt mention the more useful version of traceroute, mtr.

[@kapoork4129]

Oh yeah

[@joshshouppe5940]

In the Jeep Super Bowl commercial, The Middle, with Bruce Springsteen features the U.S. Center Chapel in Lebanon, Kansas

[@ncot_tech]

The starred connections seemed to happen during the long hops across the oceans. Does that mean it's going through some sort of undersea repeater station?

[@Lucy-dk5cz]

NCOT Technology no

[@richu2853]

❤️

[@maiaayaaoo7]

Thank you

[@takeshiasahi5494]

That was cool indeed

[@pedro_8240]

Gratuitous comment to boost engagement. Plus a duck. 🦆