Everything about HTTP Streaming and how Airbnb leverages it in production

System Design for SDE-2 and above: arpitbhayani.m...
System Design for Beginners: arpitbhayani.m...
Redis Internals: arpitbhayani.m...
Build Your Own Interpreter / Redis / DNS / BitTorrent / SQLite - with CodeCrafters.
Sign up and get 40% off - app.codecrafte...
HTTP is the language of the internet and we almost always write APIs that compile the entire response and send it to the client, but there is another way of doing it and it is called HTTP Streaming.
In this video, we understand what HTTP streaming is, how to implement it, and how Airbnb uses it to improve its user experience.
Recommended videos and playlists
If you liked this video, you will find the following videos and playlists helpful
System Design: • PostgreSQL connection ...
Designing Microservices: • Advantages of adopting...
Database Engineering: • How nested loop, hash,...
Concurrency In-depth: • How to write efficient...
Research paper dissections: • The Google File System...
Outage Dissections: • Dissecting GitHub Outa...
Hash Table Internals: • Internal Structure of ...
BitTorrent Internals: • Introduction to BitTor...
Things you will find amusing
Knowledge Base: arpitbhayani.m...
Bookshelf: arpitbhayani.m...
Papershelf: arpitbhayani.m...
Other socials
I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.
LinkedIn: / arpitbhayani
Twitter: / arpit_bhayani
Weekly Newsletter: arpit.substack...
Thank you for watching and supporting! it means a ton.
I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff.

[@devanthanki3810]

Arpit is true to the etymological roots of his name: that which is offered. He offers knowledge, joy, and nudges that curiosity faculty, which is latent in all of us. If only we could clone such people and have them teach the young ones too, be it an field of study, at the grassroots level, when they are in "alpha" and "beta" stages of their life! They would be that much more prepared to do wonders in "prod" :) Kudos!

[@sonudas7112]

Hi Arpit, great video! I just wanted to make a request. The notes you make to explain the concepts are very good. Could you please start providing these notes in your video descriptions? This will be very helpful for us to revise these concepts later. Thanks.

[@acosmicabberation8173]

I am getting addicted to your videos! :)

[@VAIBHAVVARSHNEY-p1j]

Hey can you explain how youtube manages watched videos data? how do they store every users watched data? what technique they use? how they efficiently do queries so fast on these giants data for particular user? A video on it would be gem

[@deezydoezeet]

Thank you Mr Arpit! Truly great content as always

[@akash-kumar737]

Excellent video on http streaming. This reminds me of Qwik framework 😅

[@Aditya_Vyas]

It was a good one sir. I relate with the LLMs slow rendering issue & I understand how useful this is in that section. Please keep making such. It helps learning, exploring & breaking the barriers we set in our minds. PS: Koi itna daravna pic kese rakh sakta hai yaar Github par hahaha

[@iajaydandge]

I remember while implementing SSE in Django, we struggled a lot because the response was not coming to frontend as expected. The after lots of searching on the web, we found the solution - disable caching on the proxy.

[@AsliEngineering]

Classic pitfall 😅

[@iajaydandge]

@@AsliEngineering 😅

[@7sandeepsinha]

Loved the video and the breakdown🔥 . Small suggestion - You can record [screen&video] both in 4k and upload in same, it would make reading the code snippets much easier and overall enhance the feel 😃

[@AsliEngineering]

I realized it later. will keep this mind. thanks for the pointer.

[@divyeshio]

Have to admit, I've become a fan of all the content that you're putting out. Just pure engineering stuff, which is what we want to see. Thank you, Arpit! For those who want to see HTTP streaming in action, Steve Sanderson gave an awesome talk on this, timestamped link here - kzbin.info/www/bejne/pmrXkoShe5dnfNU

[@AsliEngineering]

Thank you Divyesh! Thanks for resonating :)

[@denishclarke4470]

Hi arpit, Where can i find these notes?,

[@AsliEngineering]

I do not upload them anywhere.

[@sashpawar11]

Can this be used to load pages on a streaming website, considering the video player makes an api call that does constant streaming of chunks , which the http streaming protocal can't build late chunk of response beforehand to the client right?

[@debangshubanerjee1311]

It seems similar to reactive programming and the concept of backpressure. Can u highlight the difference if any?

[@nawed2611]

With react server components, we can just stream entire components of a page much easier

[@sahiHaiTech]

Isn't this going to be CPU intensive for server ? Isn't it better to load static components via different API calls ? Why to use streaming exactly ?

[@AsliEngineering]

As mentioned in the video, it improves UX by quickly rendering easy components first, given server can send it quickly. User need not stare at blank screen.

[@nitish5924]

This is a good optimization, but tradeoff is that it's moving all of the logic away from browser to to server, this would mean server has to have the knowledge to bring all of those data that browser needs and it would also make overall response larger.. i wonder if they apply this logic to all of their webpages or is it only for specific cases of heavy data logic ? Also correct me if am wrong.. every chunk is requested by client, as it's a request response, so the diagram of sending response from server is not one side., it's still request response

[@Adnan7887]

Well archicted video ... btw which app you use for writing on screen in your videos?

[@AsliEngineering]

Thanks. I use iPad with GoodNotes and record everything with OBS.

[@Adnan7887]

@@AsliEngineeringthanks for the info

[@rajk.4530]

Hi @arpit , is spring Flux framework is also do the same like python flaskr.

[@AsliEngineering]

not a spring user.

[@rushidesai2836]

Is this exact same concept used in HTTP Live Streaming (HLS)?

[@AsliEngineering]

Nope. HLS is totally different. HTTp streaming is different.

[@rushidesai2836]

@@AsliEngineering Thanks Arpit.

[@SEVERANCE850]

So basically Http Streaming is similar to SSE(Server Sent Events) or there is any difference between them?

[@AsliEngineering]

Nope, they are different. HTTP streaming is a general concept, while SSE is a specific implementation of it. HTTP Streaming response can also be consumed or initiated through regular APIs while SSE is typically consumed through EventSource.

[@HT79]

​@@AsliEngineering I think SSE is a really good topic for your next video given how it's still a niche concept. Can't think of anyone who can break it down as well as you would do!