The IPFS Protocol Explained with Examples - Welcome to the Decentralized Web

The InterPlanetary File System (IPFS) is a protocol and peer-to-peer network for storing and sharing data in a distributed file system. IPFS uses content-addressing to uniquely identify each file in a global namespace connecting all computing devices.
Download Slides here payhip.com/b/R1Mu
Intro 0:00
Why IPFS? 2:00
Explain the original web model and the limitation
* Content addressing instead of location addressing
* decentralized content distributed among peers
Content 3:30
* Content is hashed as CID
* Content is immutable each update generates new CID
* Content addressing
Routing 4:30
* Distributed Hash Table (DHTs) maps CID / Peer IP address
* DHT server hosts content and DHT
Publishing Content 6:30
* New Content that you want to share on ipfs
* hash the content creating new CID
* Update your local DHT CID / your ip address
* DHT will be updated to all the content peer (NOT the CONTENT)
* People searching for your CID will be connected to you and only you.
Consuming Content 8:48
* ipfs client (dht client) want to consume Ipfs://cid/
* ipfs client consults its local DHT table to see where this CID is located, gets back a collection of IP addresses
* client connects to some or all the peers found hosting that CID
* client downloads chunks of the content from each peer so it speeds up
* Once the client has the content it is now also updating its local DHT table that it now also hosts that CID (if it supports being a DHT server)
* New updated DHT is propogated across peer
IPFS Overview (Digrams) 11:30
Demo 13:45
More Information 18:30
Immutable Content
* if Content gets updated changes URI how do I inform the user?
* hash the public key of the user instead and share that
Brand new Client/server
* I know nothing about the network (Bootstraping)
* you will be bootstrapped with a collection of ip addresses to start you up.
More
* IPFS gateway
* IP Name server
* Solve content
* Deleting Content( once other node hosts it no way to delete it from their network)
NAT traversal
Resources
datatracker.ietf.org/meeting/...
• DINRG Interim Meeting,...
hnasr/status/1353...
docs.ipfs.io/concepts
🎙️Listen to the Backend Engineering Podcast
husseinnasser.com/podcast
🏭 Backend Engineering Videos
backend.husseinnasser.com
💾 Database Engineering Videos
• Database Engineering
🏰 Load Balancing and Proxies Videos
• Proxies
🏛️ Software Archtiecture Videos
• Software Architecture
📩 Messaging Systems
• Message Queues & PubSu...
Become a Member
/ @hnasr
Support me on PayPal
bit.ly/33ENps4
Join our Thriving Backend Community on Discord
/ discord
Stay Awesome,
Hussein

[@hnasr]

Download the Slides here payhip.com/b/R1Mu

[@saadbh7772]

حسين نريد ترجمة بالعربية

[@flipped_bit]

Interplanetary because IPFS is must have when humans colonize Mars: No astronut wants to wait ~44 minutes per TCP round trip on some wikipedia searches. Peers could be available on each planet

[@autohmae]

Yep, that's exactly the idea. Which is why the CID can also be a directory, not just a file. So it will be downloaded as a set of data. Would be terrible experience if you downloaded just the frontpage of a website and click for the second page and you need to wait an other 45 minutes or so to get the next part.

[@flipped_bit]

@@autohmae Yeah but you forgot the tls handshake so make that a few hours!

[@autohmae]

@@flipped_bit IPFS is just one way meant to try and solve the upper layer, I believe their is a lower layer component meant to solve the transport problem. Their is the "Consultative Committee for Space Data Systems (CCSDS)" which includes Vint Cerf (one of the "fathers of the Internet"). So when you have a normal switch on earth, what it does is: "store and forward" (Look up: Store-and-Forward Switching). What they do with Interplanetary Internet (see the Wikipedia article) what I think they do with this interplanetary communication that they do the store and forward as well. But the store part is much much longer and is even stored when not connected. As I assume you know: a communication satellite which facilitates the communication between Earth and the rovers on Mars is regularly not connected to Earth or the Rover. Because both Mars and Earth are rotating. And the rover is on the 'dark side' of Mars from the perspective of Earth some of the time. NASA did a first test in 2008 with "Disruption-Tolerant Networking" they have an article on their website: NASA Successfully Tests First Deep Space Internet

[@autohmae]

@@flipped_bit would love to know what you thought about my comment about IPF in space

[@glitchyjoe64]

good thing space isnt real

[@xxwoman]

So far I've watched many videos on IPFS trying to understand it. Yours has the best explanation. Hope you post more about this.

[@hnasr]

glad I could help thanks for your comment!

[@redjohnx07]

"Silicon Valley" TV series vibes ❤️ (#decentralized internet )

[@hnasr]

I love that show!

[@antriksh2733]

@@hnasr Me toooo

[@ryukivye]

I really like that sometimes you randomly yelled out of nowhere. Great stuff 😂.

[@ssayima]

Thank you so much Hussein. I am completely(still, for now :) ) non-tech but have been very curious to understand IPFS. It's mission speaks to me :) and I hope to host a blog with it someday and this is the best explanation I've gotten hands down! You're doing the Lord's work :) truly appreciate it :)

[@morenointl]

Thank you sir for this video. I suddenly need to learn IPFS for a client and this was I needed as a MASTERCLASS!

[@mohammadalzouabi6138]

Wow, I might work with this in my final year project

[@abo3zzzam]

Don't I tried. WAllah bro. :(

[@kevinjad4506]

@@abo3zzzam You tried what? Tried to buid something on it or just tried to build it?

[@SimeonUdoh]

I'm using IPFS to build my final year project. It's exciting!

[@ajml_hnter]

I'm thinking of using it for my final year project

[@T--T]

you always make things easy to understand! thanks

[@layla191]

Awesome Best IPFS Explanation ever😃

[@user-lq1vk5fg3h]

we knows he loves his work.

[@creativegiant148]

Thanks for making this very. Awesome explanation!

[@mathieulegrand9949]

Nice vibes! Thanks for the content Hussein!

[@yuliankarapetkov]

This is absolutely amazing! Thanks!

[@firoj.siddiki]

super awesome content and explaination. loved it.

[@user-or7ji5hv8y]

This is such a great explanation.

[@kumarcgowda]

New topics and new knowledge.

[@chenxin4741]

Hey, everyone. Very coincidentally, I just discussed IPFS with my friends yesterday. We found that dWebs/apps based on IPFS are definitely a big trend that makes data and computation more and more decentralized. However, in the meantime, big companies like Google, AWS are heavily advancing cloud computing. They are trying to make more and more running on their servers (Google Stadia is typical), etc. Which technology do you think will dominate in the (far) future? Which one is more beneficial to us? Please share your opinions.

[@carsonwyndham9693]

Thank you for this information, it is a bit above my head however I am able to grasp most of it. I am a web developer that is looking to move into the blockchain environment for hosting web content.

[@nikasekhniashvili3523]

perfect explanation, fun and exciting!

[@AllanKobelansky]

Hussein. I wish I could give you two thumbs up!

[@hnasr]

👍👍❤️

[@5150applesauce]

And a hug! ⌨️🔑🐇🕳💛👍🏽👍🏽

[@Etredal]

Thanks for the useful video!

[@sageosoro1703]

Your humor made me subscribe asap😂

[@unitydeveloperkgk9836]

Great explanation man!

[@JavierGarcia-se3vi]

very good explanation!

[@smashgaming1625]

Thanks, today I understood how torrent is much faster!

[@atriantafy]

You have a special talent in explaining things my friend

[@DeewarPutr]

If only there was an oscar for best comic in a lead engineering role.

[@Tigersoulification]

Heya m8. Good content. Just a little bit of explanation over why the naming. I have understood from them that the reason they named IPFS is because the meaning of the name as it is "InterPlanetary File System" they actually have achieved the capacity to provide connection between planets. For example, you could easily share content from Earth to Mars in worst case scenario with delays of 40 minutes and best case scenarios 4 minutes. Which as it sounds is really impressive and awesome!

[@krozaine]

Google, Facebook, Akamai, Amazon are gonna host the Peer & TURN servers for IPFS, if this decentralised architecture catches any more attention. We shouldn't forget that decentralisation is the enemy for gathering dataset for Machine Learning, corporates need cat pictures for training. Also, people are just moving from keeping less data (photos, videos etc) on their machines and pushing everything to cloud with proper authorisation which IPFS probably lacks (and will lack) Arigato gozaimasu for the content :)

[@autohmae]

IPv6 solves the problem maybe ?

[@bryang3044]

Dude, you are awesome, funny and very knowledgeable. Sign me up, subscribing!

[@sagartyagi2450]

Fascinating, curious to know more about how did they solved the hashing collision problem for the content? Multiple users can publish things with same name, will read about it.

[@contactdi8426]

So is it somewhat like the CDN clients for serving the content from nearby locations and in this case it's the every other system that's connected as DHT server. ??

[@azidanovalkran6433]

Thanks for making this video. I really liked the structure of the video, the energy in your narration, and the simple explanation. Just a question. Do search engines exist? If they don't how would you search for content without explicitly knowing the content id first?

[@hnasr]

That is a great question, I think a separate database can be built where it indexes searches and point to content id in IPFS. The database itself can then have its own contentid and put on IPFS. Although it can also be centralized

[@danielkemmet2594]

Shit dude this is such a good video. Thank you for making this! It's making getting into IPFS Development wayyyy more approachable!

[@gordz2]

Can you please explain how persistence of your Web page works? Let's say your machine is the only machine hosting the site. If no one downloads it and your computer crashes and dies then that site is just lost. Same thing for if 2 nodes have the data and both crash and die.

[@usmanaqil1765]

Please teach us how to implement microservices architecture. Thanks

[@dogecoinx3093]

Wow amazing

[@Alephu5]

Fascinating as always Hussein. This protocol has the potential to become a pillar of the web, but HTTP will always be relevant and useful. One thing that concerns me is the storage capacity of the network, if you can't delete old content how can you avoid filling up the entire address space with garbage?

[@kokas85]

You can delete old content and the content will be permanently gone. The only thing that you cannot delete is the link pointing to it even if the content is not there any more

[@JerrMakak]

@@kokas85 If I understand it correctly from the video, you can't delete it when somebody already accessed it? Presumably he also hosts downloaded content, the same erasure would have to do everyone, who also started to share it?

[@ssayima]

Very entertaining :)

[@vinitsunita]

The dealbreaker here is the implicit trust on other peers which you need to have, when you are joining in the network and hosting some content on your system. Other peers can directly establish connection to your system to download content which is kind of scary as it is posing a big security risk

[@abhishekanantharam645]

This is cool, could you make a video on LibP2P ? It is the networking stack that powers IPFS and other Decentralised projects. They have re engineered the OSI architecture by building pluggable modules instead of a vertical hierarchy. Would love to know your views on it!

[@agentpx]

yay I am using IPFS plugin for firefox :D

[@olafcreed4726]

Use Brave

[@hwkhouloud9409]

hi! can we access IPFS content from other navigator ? (eg. Opera, Mozilla ..?)

[@user-or7ji5hv8y]

Is hash of the public key the CID? I wasn’t able to follow when content is updated. What is IPNS?

[@AchwaqKhalid]

What do you think of Digitalocean managed PostgreSQL database ❓

[@mytop5602]

Can you make a video about the Hypercore dat protocol with the beaker browser?

[@tikz.-3738]

How does this work for backend systems like many ppl can give u my login page but it i submit a request how do we make sure it reaches the main computer and not others who are just having local copy of that content stored cox they viewed it or something , as far as my knowledge says something is yet to be done for that purpose coz backend systems can't really be a peer to peer like torrent and stuff is

[@abhisheksharma9796]

So basically decentralised internet would be like Torrents ?

[@newgothwhosdis]

More like torrents and git had a really fucking cool child

[@Ry-jx2tm]

Demo! arigato gozaimasu!

[@user-or7ji5hv8y]

Can someone change the content without authorization of the content creator, assuming only a few nodes held that content? Can that be an attack vector?

[@rahimnealyakoob5968]

Hey Hussein! how could i create a shared folder for different users to upload to on ipfs?

[@otc0076]

thx

[@DaWali1000]

I dont know if it works like that but, couldn't you for example upload something that is behind a paywall ( a course, a video or a document ) and make it accessible to anyone without them needing to pay for it? Wouldnt that make everyting that costs something make freely available?

[@mridul1161]

i love you ipfs

[@bachueShu]

I think IPNS is just another style of DNS. Even DNS can also resolve the domain to different hosts, can we say TCP/IP is also Decentralized Web?

[@humanperson8418]

Can you upload content that kills itself after a predetermined amount of time? That would be one way to fix the 'Deleting content' issue. You could have a private & public version. The public version could come with a 'timeout' and use the private version to re-host it.

[@dudeamir4503]

Hey Hussein! Nice content and thanks. Your claim that peer nodes once we upload the data, can download it is not correct. (bad node scenario). When synchronising, CID is not shared and without CID, no one can access that content to download. I was wondering and asked about it directly from IPFS community.

[@hnasr]

Thanks for the correction! Can you share the link to the IPFS community question?

[@dudeamir4503]

@@hnasr I asked it here on discord: discord.com/channels/806902334369824788/806902334369824793/852162525785358406 you can join their discord here: discord.com/invite/Z4H6tdECb9 Or go to their lobby from Matrix: docs.ipfs.io/community/chat/#matrix Thanks again for easy explanation of this protocol!

[@unitedmarcus8731]

We got the power baby

[@orthodoxNPC]

I'm pretty sure "ipfs cat" stands for, concatenate.

[@hnasr]

Thanks that sounds about right 👆🏽

[@GiorgiMoseshvili]

Hello, How big could be DHT?

[@hereallyfast]

Is there a way to force people's to be a dht server?

[@Mobin92]

But who actually stores that data in the end, and why would they?

[@asadd_]

Well, i want to know that if there is IPFS then why DFINITY brings ICP ? Can you plz answer and if you find it a good question, then plz make a video on it

[@userAndix]

Did anyone try opening bis link ? :)

[@shubhankar915]

I have a question : The ipfs client connects to the peer through your isp network, right? And isp is centralized, they can block ipfs protocol. How is this problem solved in decentralized web?

[@ibroschool]

It's tricky, but ipfs was used in turkey to host Wikipedia after it got banned

[@FelinePsychopath]

what I find slightly disengenuous is that Brave claims to be the first browser with built-in ipfs support, but Agregore Browser (github.com/AgregoreWeb/agregore-browser ) has had it (and several other decentralized web protocols, like hypercore protocol (hypercore-protocol.org)) for months now - and beaker browser (beakerbrowser.com) has had Dat/Hypercore support for years and initially started out as an ipfs enabled browser.

[@MaxAigner]

that's a very nice protocol. What happens if everybody just uploads their NFT data to IPFS using Pinata but nobody joins the network to maintain speed? "I want the value but I don't wanna pay for it" / there is not even an option to pay to be fair.. I am afraid this concept can only work when there is some form of payment in return for the hosting service that the peers are doing.

[@markhahn0]

consider reining in your tangential comments.

[@noahwilliams8918]

IPFS just cryptographically proves what every programmer should already know: Once you put something on the internet, it stays on the internet. Forever. Great video as usual Hussein!

[@hnasr]

Yup Screenshots ruined many careers

[@AyanMullick]

How does one buy a domain?

[@yadneshkhode3091]

There is an actual Interplanetary Highway it is the fastest path to send an object through space. For eg you want to send some rover to moon or mars you choose this path to send it in safest and fastest way

[@mytop5602]

Please make a comparison between IPFS and the Hypercore DAT protocol

[@kozie928]

The concept seems similar to hosting content on a Blockchain. You could basically upload your website to the Ethereum Blockchain, and it'll be accessible with any web3-compatible browser and truly censorship resistant

[@someghosts]

It would have to be a very small website with patient users though

[@kozie928]

@@someghosts riiight 😅 Wouldn't work out on ETH, but maybe something like Filecoin. But at that point you can just go with IPFS I guess

[@someghosts]

@@kozie928 with this solution or ipfs though, you can’t properly or easily remove anything if you accidentally dox, post api keys, copyright infringe, want to remove old info about yourself etc. Would limit the types of websites you would consider using this for. There is a great video about a guy who makes a blog using ipfs, runs into some significant issues. Some of these issues could be solved over time, but also a few issues were inherit to the way these things are designed. Would be surprised if there aren’t already solutions for decentralised websites out there already anyway. Without knowing too much about it, seems like it does permanent file storage well but not sure how flexible it would be outside of that.

[@bharatnanwani]

Isn't this similar to the blockchain technology? I don't know how blockchain works but have an idea.

[@mohamadabdelrida2866]

Having ip addresses in the DHT is kind of against the point. I think the location of the content should be somehow masked by the network. Maybe the IP addresses are hashed by the network, Also, to solve the leech problem, a coin can be awarded for hosting data!

[@madfury3179]

Is it mathematically impossible for different content to generate the same public key? There is no central authority overviewing the key generation, so uniqueness of the public keys might not be guaranteed.

[@hnasr]

You might mean hash not public key. If so yes there is a very small chance two different content will have the same CID hash. Look up birthday attack

[@cena5892]

Gigachad

[@olafcreed4726]

Won't this lead to a hard drive capacity issue if everyone is hosting the whole web?

[@jl_woodworks]

That's not how it works. You host your content which has an ID. That ID gets inserted into a Merkle DAG, the DHT. That's an oversimplification, though.

[@uksuperrascal]

Ok now how do i uninstall ipfs ?

[@simonespinedi9586]

Yes.. but... WHY? I don't want to download the whole websites of all the world... I just want to consume they....

[@ForcefighterX2]

Thanks - it's a great video! :-) But I'm not sure about the peer group for this video, though. Feels like it's intended for devs, but with almost no understanding of HTTP? It feels as if it is too hard for noobs but too dumbed down for the average dev or computer scientest. Don't get me wrong - still a great video! I just wish it wouldn't feel like Sesame Street. :-D

[@tak68tak]

kzbin.info/www/bejne/hp3ZfnqmhtOkha8 どういたしましてー笑

[@ViolaVoltairine]

are you just going to set up a separate machine for ipfs so your main computer doesn’t get hit with constant viruses? Why would anyone trust this system? And secondly, if you want to update large files like a video, isn’t having all the old versions a huge waste of hard drive space for every node who hosts it? Won’t that be a problem in the future where you need a gazillion terabytes just to host a few pieces of popular content? The more popular a thing, the more nodes and copies the more hard drive space. And harddrives are not infinite (as a video editor I can attest to this). How is that sustainable going forward? Also - doxing. Once a piece of information like a physical address, cc number, ss number, etc. is published on ipfs it can never be erased unless all the nodes holding it go offline. Seems like a recipe for great harm. As a woman online who gets regular death and rape threats, I am having a hard time trusting it. I mean sure people could hold local copies of a web page even if it’s erased from one server with http. But they would then have to republish it and that’s traceable and you can do something about it… ipfs files are like herpes. What if you accidentally download a snuff film or child porn disguised as an episode of your favorite anime series? Now that content lives on your harddrive and you are passing it around to others. You’re automatically implicated in a crime - and you are distributing shit you want nothing do with. Again you’ve got the equivalent of an STI with this. Your whole node is conpromised. How do you protect yourself from malicious or just unethical content? Nope. It’s interesting, but lots of unanswered question for me before I’d get into using it. I’ve had torrents wreck harddrives in the past. Just don’t trust it, because I don’t trust anonymous humans.

[@toordog1753]

Cat is short for concatenate

[@hackermen69]

Don't we have torrent ? why we need ipfs. Some torrent application even allows to stream video directly. Seems they have copied most of logic from torrent proto.

[@autohmae]

Torrents all have a source: the torrent tracker(s) which is per torrent. IPFS has no centralization in that way (well.. their is a bootstrap system to get a list of first nodes to connect to). And IPFS supports more complete URLs like browsers do. And IPFs supports doing updates of content, bittorrent does not as far as I'm aware.

[@rallisf1]

I see many problems here: web doesn't consist of static files any more, what about streaming? P2P is notoriously bad for that as most people still use DSL to go online. Databases? You could pair an ipfs app with a blockchain db (e.g. Bigchain) so the whole thing remains distributed but that would essentially multiply the number of peers neeeded. Unless this protocol gets wide acceptance it will just become another Tor network. Tor in my country is ridiculously slow and only used by criminals and wannabe hackers. And finally, storage; it just isn't practical for any single peer to host "the whole internet" (content could have a TTL to tackle that and delete itself from non-root peers after a while but still). At this point I don't think we need a storage protocol (let alone a P2P one) but rather an access & verification protocol for existing services. I want youtube to host my videos rather than a bunch of potentialy unreliable peers (who can't even support live streaming) but I would like to have more control over who can access my content and how my content can be proved original. This is something blockchain can do out of the box. Store hashes people, not content; that's what datacenters are for.

[@autohmae]

Well, KZbin regular videos and Netflix, etc. 'streaming' isn't really streaming (except for things like KZbin live broadcasts) it's just downloading parts of files. Look up something like Webtorrent to get an idea of how you can do 'streaming' video with P2P. It's definitely true IPFS is more beta-stage right now and is meant for wide acceptance eventually, Tor can never be fast. You might also want to check out: Progressive Web Apps (PWA), imagine storing a PWA on IPFS and PeerChat as an example of CRDT (data update protocol). From what I understand one solution could be: 'Filecoin' basically a cryptocurrency for paid storage. So datacenters could handle that part. IPFS does already have that part for verification.

[@rallisf1]

@@autohmae my concern was about live streaming, video chat etc. Didn't know about filecoin, thanks for the tip.

[@vitiok78]

Torrent on steroids

[@Saudispace]

It is considered unhealthy for children because nothing is blocked

[@CrzyGazara]

This looks torrent for the web content , it even uses the same terminology

[@jl_woodworks]

Because it's P2P.

[@CrzyGazara]

@@jl_woodworks not just p2p , the hashing technique, the distributed hashing table , basically it's missing peer discovery and you have a full bitorrent protocol.

[@nIrUbU01]

Please... get to the point.