How I Built My Portfolio Website Using React JS, Python Flask, Redis, Kafka, MongoDB & QuestDB

Let's look at how I build my portfolio website with Kafka, Redis, Python, React JS, QuestDB, MongoDB, QuestDB, NGinx and Digital Ocean.
The frontend is based on React JS. I used modern React JS with hooks to maintain states and call my APIs.
For my backend web server, I used Python Flask. The endpoints are pretty straightforward with no complex logic.
I use Redis as my caching layer. Lots of my backend calls are prone to get rate limited, so I cache the calls to avoid unnecessary calls to them. Redis makes the whole data retrieval process super quick.
Then comes Kafka. I use Kafka as the glue that connects everything. Kafka is my data pipeline. I opted for an event driven system design for my website. So every key event is a Kafka topic, and whoever is concerned with those actions, can just consume messages from that topic. I also have Kafka's retention set to infinite, so in case there is a data loss I can easily recover. That also means, if in the future I have a new use case which requires me to spin up another consumer. I can just replay all the Kafka messages and process everything from the beginning without any additional work.
In terms of my database, I am using MongoDB. I store everything in it. MongoDB is super versatile and I didn't feel like I need a relational database here.
For analytics I use QuestDB. I have a connector that directly pipes data from Kafka into QuestDB, so I don't have to worry about writing logic for it. QuestDB lets me do time series queries easily if I want to analyze how my website is doing over time.
In terms of external APIs, I use the KZbin API to get youtube videos and statistics. Also, I use ConvertKit for marketing purposes and storing user emails correctly.
As for infrastructure related things, I use Digital Ocean as the cloud platform. I deploy all my backend in a few Digital Ocean cheap instances. I am using MongoDB Atlast for remote deployment. For Kafka, QuestDB and Redis, I deploy them locally as I don't really need to scale them up for my use case.
I use an Nginx load balaner to load balance my backend and also use Nginx to serve my website frontend. Digital Ocean also lets you easily hook up domains bought elsewhere to your instances and direct them differently, depending on domains and subdomains.
Lastly, I use something called Certbot to generate SSL cetificates. That way, my website is always securely served through HTTPS.
#systemDesign #kafka #python #reactjs #questDB #nginx #mongodb #redis #digitalOcean #portfolioWebsite
My website: irtizahafiz.com
Reach me at: [e-mail]

[@codefocus]

This is awesome. Keep it up

[@irtizahafiz]

Thanks, you too!

[@vassaf1]

hi, great website! how do you fetch the posts data from redis using react?

[@irtizahafiz]

Hi! I have a backend API that interacts with both the database and the cache to return the posts data to my React frontend.

[@shuxiaokai]

Can you making a series of tutorials about this website

[@irtizahafiz]

Sure I can do that. Do you want to elaborate on what exactly you want to see me go over in the videos?

[@shuxiaokai]

@@irtizahafiz Step by step

[@irtizahafiz]

Sure! I can do something like that. I also have a few other app builds coming up in the next few weeks. You might find them interesting too :)

[@shuxiaokai]

@@irtizahafiz Sure! your videos are great

[@irtizahafiz]

Thank you so much :) Means a lot!