How Instagram Grew to 14,000,000 Users With 3 Devs

Try Snyk 💻: snyk.co/lewis
How Instagram Grew from 0 to 14M users in only a year. The best part? It was only done by 3 software engineers. In this video, I showcase the infrastructure and techniques used by the Instagram Software Engineers to handle one of the fastest growing applications in history.
SOURCES:
How Instagram Grew to 14M users post: instagram-engineering.com/wha...
Some Inspiration from Engineering Codex: engineercodex.substack.com/p/...
The Instagram engineers kept their infrastructure simple by using proven technologies like Python, Django, PostgreSQL, Redis, and more. They leveraged Amazon's cloud services for load balancing, computing, and storage. The app servers were stateless and used a connection pooler to handle the flood of incoming database requests. Frequently accessed data was cached.
MY NEWSLETTER 💌
thebetter.dev
------
CONNECT WITH ME ON SOCIAL
📸 Instagram:
/ lewismenelaws
🎚TikTok:
/ lewismenelaws
🐣 Twitter:
/ lewismenelaws
--
My gear 💻
liinks.co/lewismenelaws
-----
TIMESTAMPS
00:00 - Introduction
00:26 - Instagram's Beginnings and Initial Scaling
02:05 - (AD) SNYK
03:21 - The Role of Load Balancers in Instagram's Infrastructure
05:38 - How Instagram Handles Data Storage
08:28 - Handling Image Storage on Instagram
09:14 - Delivering Data Back to the User
10:11 - How Instagram Handles Push Notifications
10:50 - The Role of Task Queues in Scaling Instagram
11:30 - Monitoring Instagram's Systems
12:21 - Instagram's Core Principles Revisited

[@CodingWithLewis]

Hi everyone :) I got married lately and was on a bit of a break! Glad to be back and posting videos! Let me know what you want to see.

[@Youngsinsta]

Congratulations

[@user-gq8mc1wd7w]

Congratulations

[@TheZUser]

Congratulations

[@nullbeyondo]

Congrats!! :)

[@Alex.Shalda]

Congrats!!!

[@chrisnatale5901]

I really like how you turn software architecture into a compelling narrative! You earned my subscription.

[@zygimantasjasiunas5240]

Love the "True-crime stories" vibe, where all the details are uncovered step by step. Great story telling! I really like how your channel is positively progressing over time.

[@normalitee0os]

I generally don't watch such video but the presentation of this video ket me hooked to the end. Its just amazing! Kudos to you and your team!

[@heikkiliukkonen3921]

These videos are so interesting! Always the things we take for granted are somehow the most fascinating.

[@nullbeyondo]

Just watched it all! It's enlightening that you mentioned each problem they faced and what different practical techniques and technologies they used to overcome them)

[@CodingWithLewis]

Agreed!

[@pvic6959]

those 3 did that.. meanwhile me on my WFH day: watching youtube all day doing no work smh

[@user-gq8mc1wd7w]

I love these kind of videos. Keep it up Lewis ❤

[@nathanmynett6758]

10/10 As a developer in a small company it's hard to scope out what needs to be done to make a solution *infinitely* scalable. It's great to understand how big companies handle big data to provide insight on how I can develop solutions Cheers

[@iOSAcademy]

Great video!

[@JemilMarcosTyC]

Mind blown! Great video!

[@abubakarsadeeq2974]

Your contents are at another level, lewis

[@palashbaderia7113]

Loved the content, really amazing, would love to see your videos on different firm's infrastructure to understand tech better.

[@dion9795]

I love how you make engineering principles suspenseful and interesting. Well done. New sub.

[@peterakande]

This is amazing. Nice explanations, I loved every bit of it!

[@berkay2410]

Great video! Mad editing skills, Lewis. Love the vibe of this video. Felt like an episode from Mr. Robot hahah

[@realMattGavin]

I love your story time videos. They are informative and entertaining. The unicorn part got me 😂

[@davidmesaros9733]

i really like how the videos are made. keep up the good work

[@hypein2297]

Nice way to teach system design, totally liked it.

[@everythingjdm8057]

Great content man

[@avdain]

As always keep up with your videos! 😉🔥

[@benbowers3613]

Really love the format of going through each layer one at a time.

[@RajA-me9cl]

That was so helpful. Thank you.

[@zachzeurcher6450]

This format is great! Keep up the great work!

[@ahmedabd2259]

Brother your content is becoming something else, it's like watching million dollars documentary, keep it up

[@froilanimnida]

Quality Content! Keep it up Lewis!

[@illusionhex1200]

Horizontal Scaling is Awesome, I love the concept.

[@roast-salamander]

You can see the effort put into these videos. I see a great future for this channel. My subscription is yours oh great one.

[@Tharushi_SM]

This is really good. Thank you so much for the awesome content.

[@JuanesChiwirosky]

great video, it is very inspiring to see what those engineers were able to build with that three principles🎉🎉🎉

[@Marwin.]

Super interesting video. Good Job :)

[@harish.bhuvanachandar]

Great post. Subscribed to your channel.

[@maghani.official]

This video is truly incredible in its quality and content! Special thanks

[@mikulcek]

Great story telling, loved it!

[@phandc4581]

You must be one of three legendary engineers. Thanks for amazing content.

[@alexffvibes3887]

Great stuff!

[@zuman_one]

One of the best system designs video 🎉

[@sayyedhouse3871]

Video starts: "How did three software engineers alone..."

[@allezvenga7617]

Thanks for your sharing

[@supermanish4429]

i'm amazed as well as inspired to lern these tech!

[@estebanzavala9533]

knowing whats going on in the background is important even more interesting the way its being delivered here thank you

[@shis10]

Amazing Insights

[@MagnetsMediaBackup]

Very good editing

[@kunwardeepsingh127]

You should start making more videos like this, like how companies handle things 😇

[@wisdomelue]

the quality of this video is insane

[@artmort1805]

Awesome video! Thanks !

[@jenithmehta9603]

Amazing video, I'm using some of the tools mentioned in the video.

[@Pringles1]

this video is underrated

[@MrWidzek]

please more content like this

[@holthuizenoemoet591]

Amazing that Django handled the entire application sided of things. Always loved python for web programming.

[@ninobach7456]

Is Django the go to for back end development in Python?

[@holthuizenoemoet591]

@@ninobach7456 Its build really well, and has a lot of community with experience behind it. Its also easy to setup and develop with. But flask and tornado are also populair (but i wouldn't recommand them unless you work in a team that knows what they are doing. )

[@saadkaleem1449]

@@ninobach7456 Not anymore, async frameworks like FastAPI are taking over.

[@martinmetskula6384]

@@ninobach7456yes, among Flask and FastAPI

[@raizo856]

@@ninobach7456 pretty much

[@hassan.canada]

Basically the 3 engineers uses Amazon ready-to-use technologies :-)

[@Alex.Shalda]

Absolutely amazing

[@MrSaemichlaus]

Absolutely impressive how they made all of these pieces fall in line and interface correctly. I imagine it would be hell to diagnose a bug on such a complex system. Where do you even start? They would've had to choose from dozens of different terminals and log systems to even start looking at what is happening.

[@HarryDaDevSecond]

More of this!!!

[@Torbikini]

Where did you get the slick animated icons at the three rules portion? I loved those! Especially the reinvent the wheel one!

[@fabiojonathanarifin1]

i love this episode! make more please!

[@xxleosk8xx]

As always great content Lewis did you see discord went down yesterday ? can you make a video on that thank you

[@tahakhan88]

This video made me feel like a speck of dust in the field of web programming. I use php and mysql to store and retrieve info from one server. With a million users per day the way I handle the load is by making the server more powerful rather than scaling up horizontally. This is such an eye opener :|

[@davidmataviejo3313]

I love this new way to study system design

[@gawt873]

Please don't stop producing this kind of video, it's really good!😃

[@mikealejandro3938]

Nigga, this shit was crazy, and i loved it, keep it going gang 🚀

[@miguelemmara5046]

contetn is gold, its like im wathing netflix crime documentaries

[@gameplaystrailers7797]

This was sooo good so informative and entertaining Do Snapchat next

[@abhinav.t1602]

how do you create these videos? Specially the graphics that you show to explain the architecture and flow of the request/response?

[@yassinesafraoui]

How did you find these informations, I mean technologies Instagram used and may have even stopped using right now

[@gdimmortal]

my mind: Guni-corn 💀

[@pr0gstar]

Hey, thanks for the video. It was fun watching it. As a software engineer, myself, I enjoyed it very much. First you explained the challenge and I thought about what I would use to solve this, and then I watch further, and it was always the same solution the guys from Instagram came up with.

[@gabrielfono844]

Have a lot to learn as backend engineer

[@imadeddinekebour9278]

when i see this type of videos i know surely that I'm still a beginner

[@AsifSaifuddinAuvipy]

I saw a talk from Instagram engineers talking about celery framework server for background task scaling

[@iCrimzon]

Bro the scaling needed for everything is too immense to think of

[@brand5991]

Incredible production quality! Please keep it up.

[@sambegstha3516]

Brilliant engineering.

[@nothing606]

Congratulations bro🎉 Make video on AI in video games

[@CodingWithLewis]

Stay tuned :)

[@Dakappon]

Wow, this is Netflix level documentary!

[@marksantosss]

Absolutely mind-blowing! This documentary showcasing how Instagram scaled to 14M users with just three engineers is a testament to their incredible ingenuity and resourcefulness. It's a prime example of how a small, dedicated team can create monumental impact through innovation and smart engineering. Truly inspiring for aspiring tech entrepreneurs!

[@afsalmuhammed4239]

Hi chatgpt

[@the_god_killah]

ChatGPT response

[@vintiladiana-georgiana8533]

I adore this video❤

[@srijanmukherjee4658]

How does translation of object url to cdn work? Wasn't that link temporary? So it gets cached in cdn every time a new temporary object url is generated?

[@69k_gold]

Posts and messages are generally not modified quickly, so that's cool. But what about the rapidly changing ones like comments, likes, etc? How can we be sure that the cached value in the CDN is the actual value or atleast close? (Yes I'm talking about one of the two hardest problems in CS)

[@RegenerationOfficial]

are trends just data optimization for instances? because everything is just locally relevant

[@kyojurorengoku3437]

This video is beautiful.

[@claragoodlow4377]

Is there a app to make post and video

[@mistajutub2197]

I really enjoyed the content of the video 🎉 but I was a bit irritated by the flow or the narrative rhythm of the video. There was a lot of silence/emphasis on unimportant animations. Don’t want to be unappreciative, thought! I enjoyed it anyways but it’s just something I noticed ❤

[@kbsunil7464]

Every ting seems good but Infoq London summit says something different like Casandra as well

[@RandomGeometryDashStuff]

07:24 why should there be more than 25 connections?

[@xTrueBlamex]

Thanks for the great video! You spell it Gunicorn btw.. This is common knowledge. \s

[@MrIsrael3000]

Wondering why they used memcache instead of redis for DB results caching.

[@bobcoderz]

This is Modern dystopia 😅

[@user-wc1sm8cj8s]

it's crazy to think how Python powered Instagram in its early stage where it experienced massive growth, given Python's reputation for being a not so performant language.

[@IxMeTutorials]

Python still powers Instagram, however they have their own in house CPython version called "Cinder" which has much better performance and is tailored exactly to their requirements.

[@s1l3nttt]

if you made this video a while ago it would've been super helpful haha

[@Stoney_Eagle]

Thanks, I got a few clicks in my head from this story 😊

[@chrisstadler7111]

Omg! 25 instances!? Impossible 😮 😂

[@JusticeNDOU]

i think you are wrong buddy, the user application normally will have a set point of where to request the data, that set point is always a CDN . in other words all URLS on the user side goes to the CDN, then if the CDN contains the data and the data is not stale it returns the DATA, the Server Application never needs to translate the Object Storage URL to a CDN URL , its Just that the User Application is never aware of all that sort of thing or even that the Server Application will send the Object Storage URL to the CDN.

[@Fegro963]

I had to solo craft entire infrastructure for a company that is now operating in dozens of countries. When it keeps going for longer, you switch from engineering issues to mental issues..

[@surajkishor]

Make a video on how KZbin is made

[@MOHAADAM-cf1qu]

This is more useful than 4 years degree in CS.

[@entity000]

Gunicorn is G-Unicorn - stands for Green Unicorn lol so it pronounced Either GUNI-Corn or G-Unicorn depending on what you'd feel good about.

[@natgenesis5038]

As an iOS engineer ,this video is too backbend I’m kinda confused love it by the way .

[@elexbeats]

With all that complexity, resources to buy, developers to pay, have you ever wondered how the app manages to stay free of charge ? Remeber, you’re the product.

[@kyleandrade4375]

MOOOREEE👌👌👌👌👌

[@jarPlays11]

among us

[@CodingWithLewis]

sus

[@user-gq8mc1wd7w]

​@@CodingWithLewis😂