► What should I test next? ► AWS is expensive - Infra Support Fund: buymeacoffee.com/antonputra ► Benchmarks: kzbin.info/aero/PLiMWaCMwGJXmcDLvMQeORJ-j_jayKaLVn&si=p-UOaVM_6_SFx52H
@slavapol-v1553Күн бұрын
Похоже Elixir так и не дождемся
@nehua6164Күн бұрын
Great video! Could you do a comparison between Go and Swift? They're both compiled languages that can be used for backend, but Go is super minimalistic, while Swift feels more like Rust with its rich feature set. Would love to see how they stack up against eack other.
@JonCanningКүн бұрын
Golang vs Node Kafka consumer
@n3v3rd1eКүн бұрын
would be very nice to add NATS to this comparison
@MDFireX5Күн бұрын
@@slavapol-v1553 мы fastapi ждем уже полжизни
@sweetcapitan5690Күн бұрын
I think a battle between NATS and Kafka would be a good continuation of this comparison.
@TheBatmintun13Күн бұрын
Kafka vs redpanda 😊
@unom8Күн бұрын
Yes, NATS please
@PragmaticPragmatistКүн бұрын
Nats please
@praveenpereraКүн бұрын
yes NATS Jetstream comparison please
@Z3U5.0gКүн бұрын
Hell yeah I thought I was the only one who liked NATS
@x31tr0n7Күн бұрын
NATS vs KAFKA vs Pulsar vs AutoMQ should be a great addition to this series. This will be helpful for a lot of audience.
@joshuadunham495413 сағат бұрын
+1 For pulsar
@TheCarlitozgКүн бұрын
Try comparison with NATS
@jm-alanКүн бұрын
My company uses RabbitMQ as our communication backbone for an IOT-type deployment right now, so this is actually super interesting to see Also, I might have a PR - it looks like you're instantiating a connection for every RabbitMQ consumer, when Rabbit generally prefers that you try to use only one connection and multiple channels to dole out multiple logical connections to the broker. There might be performance to gain/some wasted CPU resources there, depending on how many individual consumers you're actually constructing
@Qrzychu92Күн бұрын
As usual, great video. I never expected you to add a face cam, but it gives your videos a bit more personality, good job!
@Demodude123Күн бұрын
Redpanda vs kafka would be an interesting test
@davebrooks4605Күн бұрын
I agree. This would be a very interesting comparison
@a.nk.r7209Күн бұрын
We're using redpanda in prod for our workload. Would love to see this comparison
@drksbrКүн бұрын
Yesssss!
@TheForge4718 сағат бұрын
Redpanda does not delivery what it promised, but there was a Problem with journaling File system in the past...
@joswayskiКүн бұрын
I don’t know about rabbit MQ streams, but the catch with things like Redis streams and even NATS is that you can process things out of order on the same partition or message subject if you have multiple consumers which makes it a non starter for a lot of projects As always, thanks for making these videos :)
@unom8Күн бұрын
Nice, please try NATS Would love to see IBM message queue as well
@yunocodeКүн бұрын
Thank you for the videos, very useful Would love to see some kind of websocket benching for number of connections and throughput with go, rust, js, erlang/elixir
@murtuzabagasrawala3324Күн бұрын
This is a good idea. This would be interesting to see
@artursradionovs9543Күн бұрын
Thanks for a video. Can we also check Kafka vs Redpanda?
@kronosthesoulshakerКүн бұрын
Please do a SQLite3 vs MySQL2 vs Postgres next. Thank you!
@TweakMDSКүн бұрын
Love this, I would also love to see NATS thrown into the mix.
@uwontlikeitКүн бұрын
Yes!🎉 Glad you got your talented hands on queues finally :) great job. Can't wait to see benchmarks of Apache Pulsar, NATS and RedPanda
@ttrel787Күн бұрын
It would also be excellent to see a comparison when using production best practices, being 3 replications and min isr of 2. Not sure if rabbit mq streams has an analogue but it would be really interesting if so
@PrestonThorpe-d1xКүн бұрын
I have to +1 asking for the NATS v. kafka test next!
@AntonPutraКүн бұрын
ok
@plefebvreКүн бұрын
It would be interesting to see how these two cope with slow consumers or consumer outages. One of the advantages of Kafka or Redpanda is the ability to accommodate differences in speed of processing between producer and consumer.
@wety789Күн бұрын
Would love to see comparison of kafka against NATS and/or redpanda
@AntonPutraКүн бұрын
soon
@jansyren225213 сағат бұрын
What you have in Kafka which is quite essential for many due to security is the append only and immutable lugs, the fact that they are stored to disk also retains the documents even in case of a crash. So for a banking system etc it is very important that you know the last transaction and that it isn't lost. RabbitMQ is more for less serious workloads, maybe in a web application backend but nothing I would use for anything that needs security.
@krissukoco9294Күн бұрын
This channel keeps getting better and better! Kinda my breakfast companion at this time. A bit curious, do you have Indonesian or south east asia parents? Due to the "Putra" last name.
@ericzorn3735Күн бұрын
Comparing two different AMQP message brokers such as RabbitMQ and LavinMQ would be really helpful. LavinMQ writes to disk as well, similar to Kafka
@mitchellmnrКүн бұрын
So, I do feel your comparison and wording is a bit misleading. Since both can be clustered Its not a true apples to apples comparison. Now although we do know Kafka can push more due to the way it is designed. Rabbit can handle 50k msgs per second on a single node - ive tested and seen that. But it also depends on the node specs. However when you cluster, which is what most people would do (HA, reliability and scalability) - then we can see a really good test. Although I do get having the single nodes - but that should really be said. Since its not rabbit vs kafka - its a single rabbit and kafka node vs eachother - since both are designed to be clustered
@a0flj0Күн бұрын
IME, RabbitMQ is a better choice when your messages are spread over a large number of queues/topics to which many distinct clients subscribe. It can't beat Kafka on throughput on a single queue on a single node, but IME Kafka can, in principle, manage many topics, but doesn't like it - it may unexpectedly crash nodes. If you scale your RabbitMQ cluster dynamically, based on node load, you can a humongous number of queues shared by an equally humongous number of clients with zero problems. Then again, my experience with both is a bit dated.
@sPanKyZzZ1Күн бұрын
So my take from this is running a bunch of rabbitMQ instances with a load balancer might be a more efficient and cost effective solution then kafka.
@efimovvКүн бұрын
In this configuration you have single point of failure while Kafka brokers distribute load across cluster without it.
@sPanKyZzZ1Күн бұрын
@@efimovv okay, maybe I m missing more on kafka architecture but I guess if you really want rabbitmq you can find a way
@efimovvКүн бұрын
@@sPanKyZzZ1 Sure, if someone want to use RMQ it will use despite of any tests :)
@raine-worksКүн бұрын
You should include NATS
@AntonPutraКүн бұрын
yes soon
@Z3U5.0gКүн бұрын
@@AntonPutraCLOUD NATIVE 💪
@DavidDLeeКүн бұрын
Not sure this tells you very much. Kafka and RabbitMQ guarantees are very different. Sure, if you are OK to lose data by holding it in memory, go for RabbitMQ. If you can't, then RabbitMQ will be a dumb choice. While it is interesting to see what happens when CPU usage reaches 100%, it's not a safe place to be.
@nedotraxxxx17 сағат бұрын
Did you bind stream into large queues setup or something else? I mean the kind of topology configuration that could affect the test? It seems to be weird as we easily handled 1kk message workload (RabbitMQ) with binary protocol client and there was a spot to get more. The all results are to be reasonable and expected (Kafka is definitely has maturity on a stream processing) but the numbers should be higher.
@БорисМатвеев-в2иКүн бұрын
Beautiful video, best job, sugar.! 😊
@Serizon_Күн бұрын
could you compare redwood and kafka and all other and give them points in term of specific points (like throughput , from 0 to 10) , and also a general winner when we add all the points from all metrics. This could be an interesting form of benchmark
@gobdovanКүн бұрын
are you referring to redpanda vs kafka? I would also be interested in seeing that
@AndreiBurchackКүн бұрын
If RMQ uses default settings - RMQ use some sort of "swap" for messages (which works ugly). It need to make some changes into config.
@sonk4x4Күн бұрын
Nice vídeo, it could be nice to compare also RabbitMQ vs Amazon SNS, SQS vs Azure service bus. Thank you 😄
@AntonPutraКүн бұрын
ok will do :)
@sharon8811Күн бұрын
In RabbitMQ each queue is bound to one cpu core, did you publish to many queues or to just one queue, it makes a lot of difference
@AntonPutraКүн бұрын
i created kafka topic/rabbitmq queue for each producer/consumer pair so a lot
@faressaleh3693Күн бұрын
Dude I love your videosss
@ramblexDaКүн бұрын
NATS vs Kafka would be a really interesting battle
@recklessrogesКүн бұрын
Thank you. This is very useful reference information.
@GabrielPozoКүн бұрын
Great video! I hope you will try Kafka vs. Nats!! Can I ask you how much you spent on this test?
@thomaseckert5691Күн бұрын
I would love to see a comparison between Kafka and Redpanda. Redpanda is a drop in Kafka replacement in C++. (I work at Redpanda)
@j________kКүн бұрын
Would like to see kafka vs redpanda
@StoneWeaver_RUКүн бұрын
I think RabbitMQ streams matches better with Kafka Streams over usual Kafka. And no words said about kafka partitions count
@AntonPutraКүн бұрын
well i have only 1 kafka broker 😊 you can only scale throughput if your partitions located on different brokers
@StoneWeaver_RUКүн бұрын
@@AntonPutra of course. What I meant was - kafka can perform much better in cluster mode over rabbitmq in cluster mode. You're right, it's different case
@NizamRamli-vg3ilКүн бұрын
Comparing the performance for which one is better i think not suitable. It is more sense when as a solution architect or developer to decide which broker you will choose as your solution for specific project requirement and if choose one of its you know at what time you need to scale and estimating additional cost.
@liandermedeirosКүн бұрын
Excellent content as always!! If possible, could you do Kafka vs ActiveMQ in a future video?
@AntonPutraКүн бұрын
ok will do
@ПавелПавлович-д8цКүн бұрын
НУ НАКОНЕЦ ТО! )))
@gahshunkerКүн бұрын
Great stuff🎉
@spxnrКүн бұрын
Nice! What is the dashboard where you have your plots set up in?
@marknefedovКүн бұрын
I know it is barely used by anyone, but I'm curious how Pulsar will perform.
@lufenmartofilia5804Күн бұрын
Me too !
@LtdJorgeКүн бұрын
Worse than NATS and Kafka from the benchmarks I’ve seen.
@efimovvКүн бұрын
Nice compare. But now I really curious about how topic(s) are created in Kafka. I found num.partitions=1 in Kafka config, if topic created without explicit number of partitions it basically mean single thread for producer/consumer.
@AntonPutraКүн бұрын
i have a topic and single partition per producer/consumer pair just for the test - github.com/antonputra/tutorials/blob/main/lessons/218/client/kafka/producer.go#L31
@konga8165Күн бұрын
You should do NATS
@AntonPutraКүн бұрын
will do
@spruslaks26Күн бұрын
Thanks!
@AntonPutraКүн бұрын
thank you!!
@arfathahmed3433Күн бұрын
I would love timescaleDB vs clickhouse and to make sense of clickbench.
@vani_makiКүн бұрын
I’d live to see ZeroMQ vs Kafka. zmq claims to be the fastest message broker
@mrpocockКүн бұрын
Now you've got me wanting to write a message broker :/ I'm guessing there are a ton of tricks to improve stability and throughout which would be fun to explore.
@marko95gКүн бұрын
Can you compere pulsar with kafka?
@longshin42996 сағат бұрын
Why you test Rabbit MQ with option keep msg memory but Kafka written to disk? That's is not fair. Can you test RabbitMQ ( Amazon MQ in AWS) with config mode lazy( written to disk).
@djmusti712 сағат бұрын
I wonder how this will go with the new GraalVM image for kafka.
@mustafayazlmc3973Күн бұрын
nice work
@WaseemAshrafКүн бұрын
Redis queue vs RabbitMQ should be interesting.
@AntonPutraКүн бұрын
redis queue?
@rafaeltab13 сағат бұрын
Could you also include redpanda and NATS?
@miladamiryshahrabi6503Күн бұрын
What about using a cluster of kafka brokers because no one will use a single instance of kafka in real production environments
@AntonPutraКүн бұрын
you can scale both by increasing number of brokers, this gives you a baseline for single broker/node
@miladamiryshahrabi6503Күн бұрын
@@AntonPutra As long as my knowledge helps me kafka cluster is active-active, means load and data is distributed between its nodes according to topic partitions and each partition has a different leader in cluster but rabbitmq cluster is meant for HA (active-passive cluster). a replica of master node is there so if something bad happens for master node the replica node takes its place (May be im wrong but if im correct then kafka cluster is Way stronger than a rabbitmq cluster)
@spicynoodle7419Күн бұрын
Kafka and RabbitMQ are used in different scenarios. It's kind of apples and oranges
@Qrzychu92Күн бұрын
RabbitMQ Streams are basically Kafka
@toTheMuhКүн бұрын
There is a difference between RabbitMQ and RabbitMQ Streams
@Qrzychu92Күн бұрын
@@toTheMuh yeah, and he is testing both
@dy0mber847Күн бұрын
What the difference?
@SAsquirtleКүн бұрын
FastAPI benchmarks pls!
@AntonPutraКүн бұрын
ok :)
@alexanderalexander8902Күн бұрын
Please compare NATS and Kafka
@AntonPutraКүн бұрын
ok
@kvs7720Күн бұрын
your videos all heplfull
@oztro123452 сағат бұрын
Please add comparase with NATS
@soufianosse333Күн бұрын
Can you do Kafka vs Pulsar? Thanks in advance ☺️
@dimasarestu20433 сағат бұрын
helo bro, can you compare laravel with some other framework thx
@huuphong3657Күн бұрын
please compare fastapi and golang (gin)
@arytiwa4351Күн бұрын
Benchmarking guru
@hegelwinКүн бұрын
Please compare Pulsar and Kafka
@aaliboyevКүн бұрын
Yo, interesting comparison
@cengytech497Күн бұрын
Nats vs Kafka
@AntonPutraКүн бұрын
will do soon
@chivesltdКүн бұрын
kafka vs automq!
@prosenjitjoyКүн бұрын
please add pulsar and nats too
@AntonPutraКүн бұрын
ok
@ThomazMartinezКүн бұрын
What about BullMQ?
@mateusz5216Күн бұрын
Unfortunately this test is flawed: it does not show completely what use case is, 99% of the time you'll use both for different use cases, not for performance itself. If you use Kafka/RabbitMQ for performance, simply sending/receiving then you probably doing this wrong. As a result, this is essentially comparing apples vs oranges. RabbitMQ has superstream, introduced not very long time ago, which is close to Kafka in way how it works, but it still not Kafka, it does not have Kafka guarantees and replication of data (replication is random and can't be changed by anything or key). Moreover, rabbit and kafka can be tuned for performance, but still - use case is a king here, you probably will sacrifice some capabilities that you need.
@pavanchaithanya9633Күн бұрын
Pulsar vs Kafka
@AntonPutraКүн бұрын
noted
@philipehusaniКүн бұрын
Nats Jetstream vs Kafka!!!!
@andytrooКүн бұрын
the requests per second ramp too quickly - i think that once you hit 100% cpu and the latency is building up, all you're doing is increasing the size of an internal queue - you can see that most clearly with rabbitMQ how the ram consumed just shoots through the roof - your requests per second are just being fired off without consideration of the response. once you are sending more requests than can be answered in 1 second, the completed requests per second should stop going up, even if you are sending more.
@ВалерийСидорин-я8оКүн бұрын
Eclipse Mosquitto vs EMQX
@handleking1Күн бұрын
How much did it cost?
@AntonPutraКүн бұрын
i'm too afraid to log into my aws account again 😂
@piotrpiotr1454Күн бұрын
Test redpanda pls
@minininja5505Күн бұрын
rabbit performance great before fail.
@RobTheQuant13 сағат бұрын
Great video! Please test Kestrel Web Server on Linux vs on Windows vs Node JS. Please 🙏
Why using your own expenses ? Just get a dummy email everytime and use the free credits
@OlegKorsakКүн бұрын
try Pulsar
@aykutakguen3498Күн бұрын
I am wondering how Kafka kept up at 100% cpu usage, thats wreid
@JohnEckhartКүн бұрын
Kafka is architected to push the messages to disk pretty much instantly and attempts to use the kernel’s page cache instead of a lot of internal caching. The result is that it can ingest and insane amount of data per node. The cpu in this test comes more from the consumers, so as the cpu maxes out you can see that latency increases. The other significant design in that helps it continue to operate at higher volumes is that both producer and consumer batch messages by default. So a producer isn’t ack’ing single messages, but instead it’s ack’ing a batch of messages. So even at max CPU with higher latency, throughout keeps going up because the batch sizes continue to climb.
@aykutakguen3498Күн бұрын
@@JohnEckhart OHHH
@LtdJorgeКүн бұрын
It can keep going up by trading latency, the CPUs wait a bit more per operation, but are able to handle more throughput in average.
@PonyMa-i9qКүн бұрын
Python vs C++
@AntonPutraКүн бұрын
ok
@voidwalker7774Күн бұрын
zeroMQ !!!!
@asdfsavs6846Күн бұрын
we need nuts
@AntonPutraКүн бұрын
nuts coming :)
@Takatou__YogiriКүн бұрын
Next video mongoDB vs postgresql
@AntonPutraКүн бұрын
yes i remember soon 😊
@darkoplax7688Күн бұрын
Drizzle vs Prisma ?
@krellinКүн бұрын
no one should use either of these if they care about performance and since you dont care about performance you'll probably use kafka anyway those who want performance will use aeron UDP (multicast/broadcast) for pub/sub and persis in/out queues with archive or chronicle queue... and if you do its not even comparable to kafka, chronicle alone was 750x faster and had consistent latency, i will gues aeron is even better