The Architecture of Pinterest's Time Series Database - Goku

System Design for SDE-2 and above: arpitbhayani.m...
System Design for Beginners: arpitbhayani.m...
Redis Internals: arpitbhayani.m...
Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.
Sign up and get 40% off - app.codecrafte...
In the video, I discussed how Pinterest built their own in-house time series database named Goku due to performance issues with existing databases. They focused on efficiently storing data in memory, using Facebook's Gorilla for compression, and optimizing aggregation processes. Goku's architecture includes shard-based storage in memory for 24 hours, with periodic flushing to disk. Queries are efficiently handled through a proxy server for scatter-gather aggregation. The simplicity and effectiveness of Goku's design showcase the power of leveraging existing technologies like Gorilla.
Recommended videos and playlists
If you liked this video, you will find the following videos and playlists helpful
System Design: • PostgreSQL connection ...
Designing Microservices: • Advantages of adopting...
Database Engineering: • How nested loop, hash,...
Concurrency In-depth: • How to write efficient...
Research paper dissections: • The Google File System...
Outage Dissections: • Dissecting GitHub Outa...
Hash Table Internals: • Internal Structure of ...
Bittorrent Internals: • Introduction to BitTor...
Things you will find amusing
Knowledge Base: arpitbhayani.m...
Bookshelf: arpitbhayani.m...
Papershelf: arpitbhayani.m...
Other socials
I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.
LinkedIn: / arpitbhayani
Twitter: / arpit_bhayani
Weekly Newsletter: arpit.substack...
Thank you for watching and supporting! it means a ton.
I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff.

[@Adarsh-Shrivastava]

Hi @AsliEngineering, are you sure that Bucket TS is an In Memory Hashmap? I think it should be a TSM /LSM tree, and the immutable bucket Storage something similar to compacted sorted list to make searching a given time range easier. Can you please provide any references to clarify the same.

[@anshulgupta3795]

This content is pure gold 🤩

[@ayushagarwal4247]

YT has started recommended your videos to me. Awesome. This was the first thumbnail.

[@AsliEngineering]

Finally. It's happening!

[@shervilgupta92]

Amazing, thanks for sharing this !!

[@raj_kundalia]

thank you!

[@gdthegreat]

Video treasure content

[@AsliEngineering]

Thank you.

[@adianimesh]

🙌🙏

[@onlyengineering1962]

Thanks Arpit :)