Overview of JunoDB - an open source KV store by PayPal

System Design for SDE-2 and above: arpitbhayani.m...
System Design for Beginners: arpitbhayani.m...
Redis Internals: arpitbhayani.m...
Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.
Sign up and get 40% off - app.codecrafte...
In a recent video, I delved into PayPal's open-source key-value database, JunoDB. This distributed database aims to efficiently store and cache data, reducing the load on relational databases. By leveraging Golang and focusing on high concurrency, JunoDB caters to CPU-bound loads. It offers six nines of availability, handling 350 billion requests daily for PayPal. JunoDB excels in latency bridging, ensuring near-instant replication between clusters. This video is just the beginning of a series exploring JunoDB's intricacies and architectural decisions. Stay tuned for more insights into this innovative database solution.
Recommended videos and playlists
If you liked this video, you will find the following videos and playlists helpful
System Design: • PostgreSQL connection ...
Designing Microservices: • Advantages of adopting...
Database Engineering: • How nested loop, hash,...
Concurrency In-depth: • How to write efficient...
Research paper dissections: • The Google File System...
Outage Dissections: • Dissecting GitHub Outa...
Hash Table Internals: • Internal Structure of ...
Bittorrent Internals: • Introduction to BitTor...
Things you will find amusing
Knowledge Base: arpitbhayani.m...
Bookshelf: arpitbhayani.m...
Papershelf: arpitbhayani.m...
Other socials
I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.
LinkedIn: / arpitbhayani
Twitter: / arpit_bhayani
Weekly Newsletter: arpit.substack...
Thank you for watching and supporting! it means a ton.
I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff.

[@nishankdas1895]

Hi Arpit, my question is how does Juno DB achieves this replication performance, any insights on the techniques used to replicate.

[@LeoLeo-nx5gi]

Hi Arpit, does that mean Junodb is covering all the CAP theorem properties with near/instant replication?

[@rajivsarkar277]

I don't think so because in this case if there is a network partition and the communication between the two clusters broke even though systems are available data consistency will go for a toss so 'C' in CAP is not satisfied. There is a theorem called PACELC. U can explore more about the tradoffs between various components of CAP

[@nontechnicalbaba]

There is still a chance of cache miss and req may go to db before replication. How that is handled?

[@gmmkeshav]

Can anyone tell What is the diff between CPU bound and Memory Bound plus when we will have CPU bound requirements?

[@Engineerhoon]

CPU-bound refers to a program that is primarily limited by the processing power of the CPU, while memory-bound refers to a program that is primarily limited by the speed of accessing data from memory. CPU-bound tasks are computationally intensive, while memory-bound tasks involve frequent memory access. Examples of CPU-bound tasks include complex mathematical calculations, video encoding/decoding, etc. Examples of memory-bound tasks include large data processing, database queries, and multimedia editing software that requires extensive memory access.

[@gmmkeshav]

@@Engineerhoon helpful thank you

[@koushikj6772]

you've mentioed about streaming writes in juno DB for near instant replication... how does it differ from Oracle DB to DB replication? cant we achieve same in Orable DB?

[@AsliEngineering]

Great question. But replication in a legacy db like will always be on the heavier side. Juno would have reimagined it from scratch, giving it the advantage. I am yet to go through the replication source of Juno, but I guess it is push based and not pull (like Oracle) that's my best guess.

[@pollathajeeva23]

@@AsliEngineering My guess is it may be similar to Lucene nrtReplication "Replica node, that pulls index changes from the primary node by copying newly flushed or merged index files."

[@srawat1212]

​@@pollathajeeva23Pull based replication is difficult to do in real time. Even in the lucene index replication in Solr, the NRT replicas do push based replication.

[@foobar69]

stuck in the interview hellhole...

[@rameshsmcs074]

looking forward for the series :), Thanks a lot!!

[@rkisdp]

Please make a video on google spanner.

[@amitrastogi1405]

Thanks for this video Arpit! Latency bridging is an interesting use-case using caching.

[@koteshwarraomaripudi1080]

When a client writes, it has to write to both Juno and Oracle, So should they 2Phase commit or something else ?? If they use 2Phase commit will it not impact performance ??

[@rabindradocument8934]

Why single thread model not worked for juno. Redis is memory bound and junodb is cpu bound. How juno is cpu bound, what task it performs which are cpu intensive.

[@AsliEngineering]

Encryption

[@887310954]

glad to see something in discussion what i worked on, from security aspect.

[@AkshayKumar-xh2ob]

What was the need to write a KV store from scratch when they could have used something like Dynamo db?

[@AsliEngineering]

Security, speed, and availability. Go through the series, I spoke about it.

[@jaydipaawte7324]

Enlist resources that you use,worth it

[@AsliEngineering]

Already attached in the description

[@swamysriman7147]

Six 9s means 52.56min/yr of downtime

[@AsliEngineering]

No. You forgot to divide by 100. Let me explain. Six 9s of availability" means that the system is available 99.9999% of the time. To calculate the downtime allowed, we'll need to find the percentage of time that the system is not available. Downtime percentage = 100% - Availability percentage Downtime percentage = 100% - 99.9999% = 0.0001% Now, let's convert this downtime percentage to time: Total time in a year = 365 days * 24 hours * 60 minutes * 60 seconds = 31,536,000 seconds Downtime allowed = Downtime percentage * Total time in a year Downtime allowed = 0.0001% * 31,536,000 seconds = (0.0001/100) * 31,536,000 = 3.1536 seconds More details: en.wikipedia.org/wiki/High_availability

[@gotjosh111]

Hi Arpit, nice video! What are you using to write on the screen during the video?

[@channaveerhakari7319]

He might be using Goodnotes 5 or Microsoft Onenote or iPad Notes.

[@advaitpundlik8797]

Hi Arpit , Great Content I have a question how does one calculate availability of a system ? we see at people want to target 99.99% availability and juno db has six 9's of avialibility. How one reaches that number and get a validation on there system for the same ?

[@AsliEngineering]

Measuring how much time the system was down divided by total time. It is all about having stringent measurements in place. Periodic heartbeats is one way to go.

[@advaitpundlik8797]

@@AsliEngineering Thank you so much for the reply it means a lot

[@RishabhRanjan-l4i]

Hi Arpit, Where can I find the notes you use in your videos? It will be very helpful to refresh the concepts at any time later.

[@rabindradocument8934]

Screenshot

[@pranavtiwari9948]

Thank you for the video😊

[@boomshivaa]

notification turned on!!!!!

[@raysdev]

how does Juno accomplish near-instant replication?

[@AsliEngineering]

streaming writes. talking about it in video #3.

[@raysdev]

@@AsliEngineering 🆒looking forward to it! would be curious to understand pros & cons of it or why other vendors don't or perhaps will leverage streaming writes to speed up replication.

[@harshpatel105]

Super interesting stuff

[@shubhamagarwal9511]

MMAST