Discord Stores BILLIONS of messages using this database

How does HUGE companies like Discord handle MILLIONS of users AND store BILLIONS of messages per month!?! 👨‍💻 In this video, I explain how the software engineers and programmers at Discord made the transition that made their performance SKYROCKET. Even if you don't get programming, this will be easy to understand 😄.
My name is Lewis Menelaws, I am a 27 year old software engineer who has a software agency in Ontario, Canada. I have been programming professionally for 5-6 years and am looking to help inspire you to learn code :)
MY DISCORD:
/ discord
----
CONNECT WITH ME ON SOCIAL
📸 Instagram:
/ lewismenelaws
🎚 TikTok:
/ lewismenelaws
🐣 Twitter:
/ lewismenelaws
--
My gear 💻
liinks.co/lewismenelaws
---

[@CodingWithLewis]

Thank you so much everyone for watching this video! If you want to talk more about tech, you are welcome to join my Discord channel (5k+ members)! discord.gg/bY6N9z26et How do I get Discord to notice me?! 🤔

[@opboi5498]

Hey Lewis, great video, Today WhatsApp crashed in INDIA for some time, can you dig deep into this issue and post a video explaining this, it would be interesting to know how engineers at these tech companies solve these issues in such a short time span

[@nirmalkumard7102]

Hi Lewis! I'm a civil engineering graduate(2018 passed out) but i love coding. Now I'm studying python on my own aiming to study towards data science. Is it possible that I can get a job in the data science area. Or what should I do to simply enter the IT field? Can you give me any suggestions please.

[@ertyking23]

Sooooo discord employees are seeing pretty much all the messages I sent... FUCK I'M GOING SHOULDN'T HAVE SAID THAT SHIT DAMN I'M GOING TO JAIL

[@TheMrkipas]

thanks alot for the great content matey.. would love to see video like this .. like how facebook or big social media store their data and how it works so on

[@chidiebubeiroezindu]

Hey Lewis thanks alot for all your videos they've really helped me in my tech journey. Could you please do a mobile app developer roadmap. I'd really appreciate it thank you

[@Pikachu-oo5ro]

This is what KZbin was made for. Congratulations, good sir. Bless your work.

[@CodingWithLewis]

Wow, thanks!

[@Grimzor]

ok

[@user-mx1ek4sl2m]

I like turtles 🐢

[@Ironyum99]

0:21

[@SixOThree]

Subscribed.

[@CodingWithLewis]

Thanks SO much everyone for the kind words ❤️… and the constructive criticism! My camera kept going out of focus… my mistake 😅. Also I will turn down the camera shake

[@pxtrez]

those 'effects' look cool actually

[@weiSane]

You should do more of these technical videos. Love the insights . Now I need to choose a weekend project just as an excuse for me to use Cassandra DB😂

[@sulaimandev]

Forget about Discord, think about KZbin 🤯

[@CodingWithLewis]

Literally wouldn’t be able to comprehend 😳😳

[@BatmanFailedSociety]

down

[@sulaimandev]

@@BatmanFailedSociety bruh

[@EEBDivyanshgupta]

@@CodingWithLewis a video on it??? Or probably a short

[@BlueProgamer212]

@GGLVXD bro you're familiar... are u from ISTREEM's server?

[@SomethingNick]

Would love to see more of these videos, been studying system design and this is way more interesting than the videos I’ve been watching

[@R5123]

Great video man! I'm a developer myself and this video made me realize I should be reading more articles like the one you based this video on. Great work!

[@nomadontherun_]

You lost me around <a href="#" class="seekto" data-time="297">4:57</a> but well done. By far this is one of your greatest videos. I know it takes a lot to put these together. Again, great job. Keep up the hard work.

[@john95685]

Amazing video 😍. Being a beginner in software development, It shows me how much is there to explore in this field.

[@ShafterPlay]

Really great video! Loved how technical but easy to understand it is. The camera shake was a bit too much for me tho.

[@COOLUKGAMER]

Loved to watch this video. I remember using Discord as a reference for an A-Level project I did and I ended up reading their development blog

[@kristefanov]

Great video. Would recommend diving into a Spotify-based topic. Their engineering blog is quite rich with similar examples and could be a good platform to explain basic concepts from.

[@stuiedaman]

Oh hey, it's one of those times KZbin recommends a channel I've never heard of before but turns out to be really good educational content. *Subbed*

[@onverion]

Insane video! I found you through your Shorts but I love the longer videos!

[@saramshshrestha7641]

Keep more videos like this coming. Loved it

[@mostlywhat]

This is the style I like! Keep up the good work!

[@-_-_-_-_]

I really like your longer content!

[@CodingWithLewis]

Im glad :) they take sooo much longer to make but love doing it

[@-_-_-_-_]

@@CodingWithLewis But then again it's much more profitable :D

[@jannatgaoshiqqalb3598]

@@CodingWithLewis Bro, what fundamental knowledges do you recommend before learn to code?(my aim is to get into big tech companies and become a good software engineer)

[@hand-eye4517]

I love the way your content is made , flows well , gets the points of the topic clearly and concisely . Just had to give you the credit you deserve really quickly and also mention that i haven't been seeing notifications for your videos? strange ill just check the discord more often .

[@eurotrucksimulator2741]

Nice video! Discord is insane. This video reminded me when like 6 months ago discord was down for 1-2 hours.

[@dailydoseofhalal6767]

Love this type of video! Nice work :)

[@ash_ekka]

Now this the content KZbin should push for. Pure gold ♥️

[@shivankchaturvedi210]

Wohh! Absolute bliss information I m data engineer and let me say you just described our months of work in just 10 mins very effectively . Nice video keep it up:)

[@INKILU]

always wondered about this! Thanks for making a video about this 👍🏼

[@CodingWithLewis]

No problem!!

[@urvishjaiswal8222]

Beginner friendly and easy to grasp lewis❤

[@eshaannilekar3851]

Amazingly explained!Great graphics and attention to detail.. Make more such vids like this

[@jordanfarr3157]

You called out the YT algorithm for bringing me here, and uh, yep! Great video! I have been a programmer for 10 years but managed to only skirt around the sides of the database world. This helped clarify that a bit more!

[@iamcrabgod2809]

This video is gold. Though I wanted you to go more in-depth into this , but I liked it. I want more of this and please go more in-depth. I would love it.

[@SeanCassiere]

Love these videos on "tech stories from companies".

[@GMoyo]

This was awesome, I’m subscribing rn

[@jaskidesigns8355]

Woah, great video mate!

[@ashharkausar413]

Brilliant stuff, ty for sharing!

[@PriyankRupareliya]

Beautiful. Too much information in too little time. Pure gold

[@grimjetram3866]

this was really really nice bro as an aspiring backend developer seeing stuff like this and knowing how it works on the backend its amazing lol i was glued to the the screen from the start thanks for an amazing video this makes me push more towards my goal of becoming a backend developer thats just amazing💯💯💯💯💯keep up the good work

[@abhaysingh.632]

Very informative insights, and I subscribed 🎉

[@groundingtiming]

Def nailed this vid Lewis. Follower from the start, you sure grew quick ! Talk about scalability huh!

[@kacperkobuc9582]

Very interesting video, and the jokes are just a cherry on top!

[@MrDesai]

I am a mechanic engineer and I still enjoy your informative videos Thank you

[@lelouch9609]

This content what I want Thanks Lewis ☺️

[@Marwin.]

This was amazing. Thank you 🙏

[@CodingWithLewis]

You're so welcome!

[@meu2121]

It's amazing!! how u people are helping the whole world . Thanks for ur existence ☺️

[@g4_684]

This video is amazing. Loads of information.

[@JonBeeee]

Very informative, thank you!

[@allezvenga7617]

Awesome video👍

[@Tnvrahmed98]

Subbed, great Content.

[@DiOnFire]

Amazing video! 👍

[@cesarlabastida1392]

Great video everything well explained

[@UCezI1mEFSJR_GHO4Z7bs8YA]

Great video dude.

[@tusharverma4202]

Loved it Lewis, thank you so much!

[@kephiob]

👆send a direct message for support and guidance . . .

[@artemisfowl127]

This was an amazing video, I'm a SWE myself, and everyday I've used this damn god send of an app, I've always wondered how it works so well with millions of users, servers, channels and billions of messages. Do ICloud next!

[@samuelstefancic8050]

Idk why i am thinking so much about this video, what a amazing work you've done here.

[@kephiob]

👆send a direct message for support and guidance . . .

[@connorradeloff5597]

Great video! I just found your channel, and I'll for sure be going down the rabbit hole. If you're willing to accept some feedback, I would consider significantly reducing (or IMO getting rid of) the amount of simulated camera shake in your profile shots. I personally found the amount used in this video just tolerable, but it was becoming hard to watch at some points. A little goes a long way!

[@JustPyroYT]

Great Video! 👍 Greetings from 🇩🇪 :D

[@lakshman587]

Thanks for the video!!!

[@M-ABDULLAH-AZIZ]

Amazing explanation mate❤️

[@kjl3080]

Really cool video!

[@MisterValiant]

Videos keeps getting better

[@codewithguillaume]

Soooo Interesting :)

[@HeyFaheem]

Good Way Of Explaining Sir....

[@DevNetic]

Interesting video. I had this question also for a really long time but never took the time to look into it. Now I have the answer

[@_Nonines]

Great video!

[@SportSync_official]

great video!

[@guillaume5623]

This is gold!

[@kleptotrichy]

Very informative video showing the ins and outs of a mega data management platforms!!!! I do still wonder what the hell did discord do for their september release on android that made it so much buggier haha Good video overall!

[@trylaarsdam]

Afaik they switched to react native from native apps which caused a lot of issues

[@estebanduvanzzDelgado]

You tell the story with so much joy while I as an engineer imagined a rollercoaster of tears, frustration and susfpense in the face of each one of the engineers working to deliver the migration. As you correcly said. DB Migrations are the worst of our nightmares

[@kephiob]

👆send a direct message for support and guidance . . .

[@vitorm7236]

Excelent video!

[@mallikarjunhj8768]

Would like to see more of how they built it

[@zhehuizhou]

Have to point out that it's "sea-lla" instead of "sky-lla". Chatted with folks from ScyllaDB, and Discord is pretty much on Scylla now.

[@bhaveshagarwal2795]

<a href="#" class="seekto" data-time="152">2:32</a> no pun intended 😂

[@gabbyes]

nice video bro

[@mushroomcraft]

Back in the day, when Discord engineers focused on improving the platform instead of ruining the platform...

[@Lark703]

I’m out of the loop. How are engineers ruining the platform these days?

[@mushroomcraft]

@@Lark703 Well, many ways like the new Android version which is horrible, and checking if your bot's description in the app directory contains "youtube" to unverify it.

[@OatmealTheCrazy]

@@mushroomcraft I'm unsure what's wrong with the new Android version other than maybe seeing too many image previews at once when trying to upload

[@mushroomcraft]

@@OatmealTheCrazy In a technical sense, it is just the IOS version slapped onto Android, which means that it won't be optimised for Android making it slow and it has lots of stupid problems like I sometimes have to wait up to a couple of minutes for it to connect. None of these things were issues pre-new-android-version

[@chlorobyte_projects]

bruh, my phone's specs are like close to top of the line, yet there's this one single gif that makes Discord run at 10 fps. Not even a problem with the gif itself, Firefox and the Photos app handle it very smoothly. Nice "native performance"

[@FinickyHorseGameStudios]

Very interesting!

[@Viralvlogvideos]

Wow. This video is awesome. Please make a video on hotstar stack, It was really amazing to see hotstar handling more then 1crore streamers at once during cricket match.

[@Sammysapphira]

Puzzle & dragons lmao I loved that game

[@bob_pm]

I really liked it! Thanks. Could you make another video but with the Telegram.

[@bpoole007]

I loved this one.

[@jakeh5852]

Engineer here who worked on these systems. A lot has changed since 2016. We don't even use Cassandra anymore. We should probably write a new blog post. 😂

[@vigneshshanmugam]

Hi Bro, I learnt how Big data to be managed ❤️🔥 keep doing and rocking Love from 🇮🇳

[@CodingWithLewis]

Love from 🇨🇦

[@stevepoper8073]

Can't believe an app that I use to frequently is so incredibly complex

[@愛]

"Wow guys look how cool, Discord is logging all of my private messages and everything i say!!"

[@jannatgaoshiqqalb3598]

Hi Bro, keep going Love from Uzbekistan 🇺🇿 (whose grandfather is founder of Algorithm)

[@ZombieAssassin7777]

With MongoDB going through many major changes over the years, I wonder if Discord has tried to go back and test the performance of it today.

[@javierflores09]

yeah, no. You don't go back to something that works and has been deeply integrated in your infrastructure like this. They either use Cassadra or fix Cassandra but that's what they got for the end of the times

[@tonyhart2744]

mongodb is child toy company, if we talk about planet scaling we talk with vitess or cassandra

[@ZombieAssassin7777]

@@javierflores09 That's blatantly false. You're never stuck with something till the end of times. In the tech field, there's always room for improvement.

[@javierflores09]

@@ZombieAssassin7777 developers make improvement, companies don't necessarily go along with them. In production you are not changing your underlying infrastructure unless you have a damned good reason to, just look at banks still using Cobol; if it was a startup which isn't yet as established I'd understand but something like Discord can't really handle the weight of these changes anymore, so yes, you are in fact stuck with it

[@ZombieAssassin7777]

@@javierflores09 Still wrong, I work at a bank. Yes we still use Cobol and Mainframe, but are working towards moving off of them and also in the process of changing our database configuration.

[@Verulyrus]

this is seriously interesting, as a developer and discord user. I've asked the question 'how the fuck do they do this all?' many many times, and you gave a well explained answer! Amazing work!

[@Kat_Boyy]

You should do twitch next, how they moved from REST api to GraphQL in 2017

[@caiocesarmelolopes2156]

GitHub! Me as a backend dev i am facineted how the github can do what they do 🤩

[@suharsh96]

@<a href="#" class="seekto" data-time="505">8:25</a> You sure about that? AFAIK reading is much more computationally expensive than writing to a DB, especially in NoSQL databases as the indexes has to be updated when data changes in one place.

[@piyushsahu3841]

Want more videos like this

[@xslvrxslwt]

I already knew this since the very first day they switched to Cassandra but still watched the video, lol.

[@YashSharma0605]

Crazy insights 😵😵

[@kephiob]

👆send a direct message for support and guidance

[@phil7121]

So this is why tech companies ask interviewees to invert binary trees

[@givemeyourmangoes5040]

I love the way you broke it down but the artificial camera shake was a little disorienting since i was eating while watching Love ur content tho!

[@MainActor]

I mean we only have about 300k users in our website & little app combined, and we’re having all sorts of issues with our database, and we’re struggling to keep the thing up and running 😂, let alone having 150 million using your service, it must be a challenging day to day task for the engineers at Discord.

[@potsuaye69]

they're doing these amazing stuff yet im using their technology to send memes in general

[@silvankuebler]

Reading is less expensive than writing in a DB :) This is due to the fact e.g that reading can be concurrent with no problem. You said it the other way around

[@TheHasanJr]

This was soo-oo great! Keep making contents like these. Try WhatsApp or Facebook or KZbin next please!!!!

[@postnick]

Thank you for exposing how Cassandra works!!

[@CodingWithLewis]

No problem!!

[@dvs6121]

<a href="#" class="seekto" data-time="458">7:38</a> seems like you can test the collision condition before the db reaches scale.

[@bhaveshagarwal2795]

great explanation

[@CodingWithLewis]

Glad you think so!

[@chidemenot]

Great dissection & insight, inside the tech details Buddy, Thank you

[@kamertonaudiophileplayer847]

Surprisingly, but paperless receipts use a very similar solution, because they operates with even bigger data.

[@Valery0p5]

"and what if I fill a bucket with tombstones, wouldn't it lead to resource exhaustion?" 5 seconds later "Knew it "

[@Valery0p5]

KZbin, you are the only half decent social network left, these bots are so easy to see, please...