System Design Meetup 2021-Jul-16: Kafka
1:27:21
Пікірлер
@scottcorley8448
@scottcorley8448 4 күн бұрын
I'm from California. I'm looking for this video for the last few days. It was very helpful for me.
@davonowens5053
@davonowens5053 5 күн бұрын
Thanks for your valuable explanation. Keep it up 💪
@CydneeCintron-ht7sd
@CydneeCintron-ht7sd Ай бұрын
This video is very useful for me 🙂 really thanks for making this kind of videos 🤟
@DRaleigh88-gv5uh
@DRaleigh88-gv5uh Ай бұрын
👏👏🙏
@NhuleipoKatale
@NhuleipoKatale Ай бұрын
I'm looking for this video. Thanks for sharing
@LisaDubeau
@LisaDubeau Ай бұрын
Thanks Dima
@JuanNavaro-i2h
@JuanNavaro-i2h Ай бұрын
Wonderful explanation! Keep up the good work.
@EstrellaMartinez-rv3xc
@EstrellaMartinez-rv3xc Ай бұрын
Keep it up
@MerlyRoque-jc4cp
@MerlyRoque-jc4cp Ай бұрын
🔥🔥🔥
@ImangaLutombiJr-eg6sj
@ImangaLutombiJr-eg6sj Ай бұрын
Bohot jada beautiful explanation ❤🙏🏻
@CalebHale-xd5jf
@CalebHale-xd5jf Ай бұрын
Thank you dear. It was an amazing yet simple explanation. You keep us connected to your channel. :)
@NadiaVenice-up7pe
@NadiaVenice-up7pe Ай бұрын
Make more videos of this kind. Thank you
@teddylane7890
@teddylane7890 Ай бұрын
Very informative video. Thanks for sharing 🎉
@LukePatrino-mv8me
@LukePatrino-mv8me Ай бұрын
This is the best explanation I ever listened thank you for making this
@dimakorolev4740
@dimakorolev4740 Ай бұрын
Thank you!
@shanybiran
@shanybiran Ай бұрын
Can i use orchestrations for faster answer? like i want to create chat bot and my endPoint(httpTrigger) will receive command then i send this into orchestration function then i use trigger for calling command function and wait for answer, but it doesn't work fast, why? i don't understand
@dimakorolev
@dimakorolev Ай бұрын
How fast do you need it to work? Orchestration is generally for workflows, not super-fast responses. If you need real-time answers, and are fine with some dropped requests, a simpler solution is often better.
@LaQuianaGaines
@LaQuianaGaines Ай бұрын
This is mindblowing!!! What TR is this on and do you have any documentation links please
@KodjoTyrone-yx3hz
@KodjoTyrone-yx3hz Ай бұрын
Great video
@TrueTimrus
@TrueTimrus 2 ай бұрын
@dimakorolev, hi, why is using UUID for message_id from client bad? it is essentially a representation of the same 128 bit int
@dimakorolev
@dimakorolev 2 ай бұрын
Yeah, there were plenty of posts on this. TL;DR: If the underlying DB stores it as 128 bits, there really is no noticeable harm in storing the UUID directly.
@TrueTimrus
@TrueTimrus 2 ай бұрын
@@dimakorolev ok, thanks!
@aprasath1
@aprasath1 2 ай бұрын
@dimakorolev I totally agree with your views on 43 minute mark. Bascially we need to take care of consistency of lot of things, database only handles a small portion. The way I see some banks design this is 1) have the receiver in the pre-approved receiver list 2) check the balance is present(might be ok now, but might fail while actually doing the transfer) 3) queue it for execution(either sucess / failure) 4) notify. Item 3 is the only piece of transcation that RDBMS would do, all others are handled at business level.
@programming6881
@programming6881 2 ай бұрын
Awesome video.
@programming6881
@programming6881 3 ай бұрын
@Dimakorolev awesome video, even though it was not much of a discussion.
@dimakorolev
@dimakorolev 2 ай бұрын
Yeah, I'm trying hard to make it more conversational! Happens a lot off-the-record, but the ones we record appear to mostly be me speaking still 😞
@programming6881
@programming6881 2 ай бұрын
@@dimakorolev thanks for lot for posting this content. I am preparing for my system design interviews at Google and Microsoft. I am never a fan of this kind of interview. Your videos are explaining the thought process behind it. I read klepmann's book and, also covered the fundamentals of distributed computing. Your videos are where I see everything tied together. Design auth , exactly once are all wonderful topics. I am going to watch all the content here. I am moving my interview by a week to cover all the videos.
@markeno5067
@markeno5067 Жыл бұрын
Is there any way to join with you?
@dimakorolev
@dimakorolev Жыл бұрын
tinyurl.com/sdm-slack-invite
@annaheller900
@annaheller900 Жыл бұрын
Detailed session. Since system testing involves a multitude of feature to be tested. Its require a lot of time and effort. Expenses come above this and that's where an automated testing platform makes a difference. Avo assure like platforms make system testing a seamless as possible and offer multi-benefits, including cost and time.
@jorgemonroy7234
@jorgemonroy7234 Жыл бұрын
Thank you for all the informative videos that you make.
@JuanNavaro-i2h
@JuanNavaro-i2h Жыл бұрын
Thanks a lot for this video. This is such a detailed explanation for the people who are beginners in the software testing.
@muddassirbari
@muddassirbari Жыл бұрын
how can I join the dicord server?
@dimakorolev
@dimakorolev Жыл бұрын
tinyurl.com/sdm-slack-invite tinyurl.com/sdm-discord-invite Our Slack is by far more popular, but I can accept the possibility of this changing in the future.
@JuanNavaro-i2h
@JuanNavaro-i2h Жыл бұрын
Good job. Keep it up 👍 We want like this useful videos in every week..
@dimakorolev4740
@dimakorolev4740 Жыл бұрын
I wish there were enough hours in the day. We're trying though, thank you for your encouragement!
@abhilashkr1175
@abhilashkr1175 Жыл бұрын
How can i join the next meetup? Any discord group?
@dimakorolev
@dimakorolev Жыл бұрын
does tinyurl.com/sdm-slack-invite work?
@dimakorolev
@dimakorolev 11 ай бұрын
Also, there's tinyurl.com/sdm-discord-invite now!
@scottcorley8448
@scottcorley8448 Жыл бұрын
This was very useful and informative but also seemed to present and extremely optimistics view of web 3.0. It can't all be the rainbows and unicorns and enhanced security. But thank you...
@dimakorolev4740
@dimakorolev4740 Жыл бұрын
Heh, technologically I do believe Web3 is in good shape. Whether the regulatory frameworks or the free-spirited individuals prevail in the longer run remains to be seen.
@davonowens5053
@davonowens5053 Жыл бұрын
I usually used to prefer short video but wanted a deep understanding of the whole thing.so started with this and felt in love with it.
@dimakorolev4740
@dimakorolev4740 Жыл бұрын
Thank you Davon! Recording shorter episodes is on my TODO list all the time, but looks like it does work better when I expand on the ideas in depth. Hope we can get the best of both worlds one day!
@8Trails50
@8Trails50 Жыл бұрын
Came back to rewatch this talk. Still as good as ever. There's a lot of information packed into this.
@dimakorolev4740
@dimakorolev4740 Жыл бұрын
Thank you. This topic is something I'm pretty knowledgeable in, and there was quite some time invested into preparations. Glad to hear it's worth it for you!
@saber3112
@saber3112 Жыл бұрын
welcome back
@vojindjukic2719
@vojindjukic2719 Жыл бұрын
Great presentation, thanks for sharing
@dimakorolev
@dimakorolev Жыл бұрын
Much appreciated, stay tuned for more!
@asyavorobyova2960
@asyavorobyova2960 Жыл бұрын
Доброе время суток! Скажите пожалуйста, есть ли возможность присоединиться как-нибудь к вашим митапам?
@dimakorolev
@dimakorolev Жыл бұрын
tinyurl.com/sdm-slack-invite
@asyavorobyova2960
@asyavorobyova2960 Жыл бұрын
@@dimakorolev Спасибо!
@davonowens5053
@davonowens5053 Жыл бұрын
Thank you for this information and clarity...
@ginariley434
@ginariley434 Жыл бұрын
Awesome! Straight to the point. Thank you
@scottcorley8448
@scottcorley8448 Жыл бұрын
Thanks! it's much easier than people make it sound.
@tonyholis9260
@tonyholis9260 Жыл бұрын
What a great video. Very easy to understand. Thanks for the explanation
@harshtrivedi29
@harshtrivedi29 Жыл бұрын
The slack join link is not working, can I get a new link, pretty please :)
@asyavorobyova2960
@asyavorobyova2960 Жыл бұрын
Все хорошо, но как по мне не хватает картинок, тогда алгоритмическая часть была бы сильно понятнее, и можно было бы говорить вполовину меньше ;) Как математик по образованию, знаю о чем говорю.
@dimakorolev4740
@dimakorolev4740 Жыл бұрын
Будем работать над улучшением качества ;)
@markeno5067
@markeno5067 2 жыл бұрын
Great work. keep it up
@travisvarga4511
@travisvarga4511 2 жыл бұрын
I love this discussion
@scottcorley8448
@scottcorley8448 2 жыл бұрын
Keep it up 👍
@davonowens5053
@davonowens5053 2 жыл бұрын
Great discussion...
@scottcorley8448
@scottcorley8448 2 жыл бұрын
❤️❤️🥰
@davonowens5053
@davonowens5053 2 жыл бұрын
Good job 👌. Keep it up
@ginariley434
@ginariley434 2 жыл бұрын
It's very helpful content. Thanks for post
@8Trails50
@8Trails50 2 жыл бұрын
one question: ive heard of both actor model and coroutines separately, but never together. any examples I can see this from?
@dimakorolev
@dimakorolev 2 жыл бұрын
In reality, even though actors and coroutines don't technically contradict one another, the high-level design decision is generally one or the other. Either you bet on being lock-free, and emphasize each actor having its own state ("you don't communicate via shared memory, you communicate to share memory"), or you bet on high performance via high concurrency and handle possible (and tricky!) multithreading issues yourself. That's the theory. In practice, one is 99% likely to be using coroutines when they are using actors. The coroutines themselves would just be hidden deep enough in the actor framework they use, be it something external or something small and home-brewed. Keep up with great questions, and thank you!
@8Trails50
@8Trails50 2 жыл бұрын
@@dimakorolev thank you - that makes sense. Great video as always!
@prathikh
@prathikh 2 жыл бұрын
First of all, I really like the idea to discuss these concepts in a meetup. On a separate note, at 58:08, it's mentioned that the page should probably be crawled from the region where it's hosted. My query is, how does one achieve that? Would that not require one to know where the page is hosted and it's not just a limited set?
@dimakorolev
@dimakorolev 2 жыл бұрын
That's actually a big question, largely because most pages these days are served by a CDN, so the very domain name would resolve to a different IP address and the request would to different servers accessed in different parts of the world when queries from different places. For the simple case, it's easy: resolve the DNS name from different datacenters, if they return the same IP, geo-locate it and/or try crawling from different places and measure the throughput/latency yourself. For CDN-served content, I'm on expert, but would imagine there is a way to request the CDN to return the best location to query it from (I would want it to be a primary location, so that it's updated faster, and my crawler catches fresh pages sooner). On the other hand, we may just ignore this part altogether and rely on our own algorithm that tries crawling pages from different datacenters and gives preference to the one that responds with content faster. Last but not least: the server may return different content for crawl requests from different places. This is also something to be factored into the crawler algorithm.
@evgenf9677
@evgenf9677 2 жыл бұрын
great episode, as usual, thx