Java Concurrency Interview: Implement Scatter Gather pattern

[@mattj65816]

Please keep making these videos forever. They are wonderful for sending to junior developers who haven’t learned these things yet. You must have the most comprehensively “correct” fundamental Java tutorials on KZbin. Case in point, your use of a synchronized set and a countdown latch here. They are not core to what you are teaching, but you use them anyway. I think most Java demos would use a HashSet and an int there, and it would likely work in a synthetic demo environment, but would yield insidious bugs in real life concurrent use. Thank you for your efforts.

[@8Trails50]

Man. These videos are just wonderful. I’m a junior developer who has desperately been trying to get better at concurrency and distributed systems. There really is a select few people breaking things down like this. Perfect mix of theory and practical knowledge. Thank you!

[@SushilKumarBhaskar]

One of the excellent explanation(EE+) found in you tube. 100000+ incremental Likes...

[@tusd100]

One of the best content w.r.t multithreading in java

[@jagannathsahu1087]

Thanks. Everyone should go through these types of lessons to get more understandings on concurrency patterns

[@SatyanarayanaBolenedi]

Wow... Man, I'm in love with your concurrency playlist!! Your explanations are so crisp & clear. Thanks a ton for making these videos!! I feel like giving a different title "Conquer the Concurrency in JAVA"!!

[@mohitdarmwal7309]

one of the best content and work on concurrency, Keep the Good Work and a very thanks for the information :-)

[@anubhavkumar2246]

These videos helped a lot in starting my job as SDE-1. I am truly indebted to this channel.

[@theunusual4566]

for m=1 to infinity Thanking you for Explaining the concepts so Amazingly

[@pdteach]

Covers quite a topic in one go. Thanks

[@ShivSinghSengar7013]

Gr8 way of explanation better if I had visited here than reading concurrency in practice first

[@ILuvBilli]

I have seen CoundownLatch many times but first time understand since your way explanation is too good

[@shrikantarbat2506]

How someone cn dare to dislike such informative videos...:(

[@spinningbullet9136]

I use this for timeout invokeAll(Collection

[@quantum634]

From the beginning I knew you could solve this using CompleteabkeFuture's with .allOf, but after seeing the 2nd solution with CountdownLatch I find this more "elegant" and cleaner. Loving your videos. Keep up with the great work. Devs like me, can learn a lot of these type of tutorials. Thank you for putting so much work into it.

[@freemeplzplz]

Your videos on concurrency are really amazing. Please keep adding them.

[@MadhuCM]

Brilliant. Thanks for giving this wonderful session and I really appreciate that you are spending your time that benefit others.

[@satyanarayanajammala5129]

You're awesome boss. The best explanation I have ever seen in KZbin

[@ragingpahadi]

Love your videos ! Please keep them running

[@Sambitsabyasachi00]

Thanks for making wonderful videos and plz make some videos on Spring security and Oauth2.

[@LawZist]

I will be more then happy to see new videos from you :)

[@ShrinathKuppa]

Thank you for this video. Really helps understand the concept better!

[@ruhirc]

good explanation, it will be great to see exception handling too.

[@AbhishekVaid]

I think passing accumulator (although ThreadSafe) has a problem here. My thought process is that after 3 seconds, we are returning prices from main method. But the reference of this method is still with a running thread which might add another entry to the prices after the return statement. For me, this volatility might cause problems. A safe way would be to make a copy of the accumulator and return that value.

[@matthewsaucedo2471]

This is an excellent point!

[@dineshchandgr]

Amazing Video. Great clarity and communication

[@lukevin3833]

Thanks, that's a good video! But I think there's a problem in code at 7:00 Let's say there is one I/O thread cost 3. 1 second. The "business thread" executed code "latch.await(3, TimeUnit.SECONDS)", and execute other code over one second (but not finish yet), then the "invalid data" will still be settled into the set. So, I think we'd timing 3S in every I/O thread too. ps : I'm not good at English, sorry about that

[@DefogTech]

I think only problem is we are returning same set object which is still accessable by tasks. So we should instead return a copy of the set. Also, using a simple set used by multiple threads is not best. We should use some synchronized collection instead

[@SatyanarayanaBolenedi]

@@DefogTech Hey, I see just one price in output [0]. Not sure, why Task.prices is copying/appending old prices?? I've some doubts around how do share data structures across threads like this. Highly appreciate, if you can shed some light here...!! Thanks in advance!!

[@neerajsharma8876]

Thanks for making wonderful videos on concurrency .please make some videos on concurrent collections and stamped lock.

[@souravsarker9100]

Dude, awesome explanation. Really appreciate your effort for making these valuable videos.

[@hreddy]

Super video! I applauded for ₹40.00 👏

[@neerajbhatia2008]

Awesome explanation and great presentation in all the videos, keep up the good work. I am subscribing to your channel now, I think CountAndLatch example impl might change if there are like more than 500 websites to gather price from and some of the threads have not yet started their work like change into runnable state so may be we need three latch in that scenario one for ready, one from main latch when all threads are ready and third one for completing the run(gathering all the prices) then await for completing latch.

[@madnorbi]

Excellent! Small note: it might make sense to also handle the possible TimeoutException. Say "allTasks.get(3, TimeUnit.SECONDS)" times out. In this case the intended action is to still return a partial result instead of propagating the TimeoutException.

[@jeremythen7239]

That was wonderful man. Keep it up. You are the best.

[@IlyaTkachev-hs8cb]

Thanks a lot for your tutorials!!!

[@sadanandburud5157]

Incredible explanations you provide in all your videos. I really amazed by your explanation. And one best positive thing is, you make it easy and for others it's easy to grasp and which stores the concept in the permanent memory. :). I have one question related to this video...... If thread doesn't perform the task in that particular time period which was set in await or any other method, what happens to that thread, will it get killed or do we need to send the interrupt signal..? Or will it get killed by itself.. if doesn't kill, and if consider it's doing some memory processing logic as well. Then havoc situation will arrive. Could you please reply what exactly happens..

[@gautammishra3436]

Awesome explanantion . Why are you using Set for prices? Cuzz prices can be same in that case we will not get all required result

[@sharanyarai378]

Perfect ! I Your time and efforts greatly appreciated.

[@ChrisAthanas]

Excellent explanation

[@mostinho7]

Done thanks took notes

[@virendrashinde304]

awesome explanation.

[@sambhudevware7262]

Is there videos for concurrency collection such as concurrent HashMap implementation and internal working

[@rohithvarma2311]

Amazing videos....can you mqke videos on spring security ?

[@javacodingskills5525]

Great explanation bro :)

[@anckursingh]

Workstealingpool could have been used? Or fork join pool explicitly?

[@jakubcichy5220]

very good chanel grats!

[@kailashpatel223]

A big fan of your work.. I have a problem statement: We read from a big file and write data in batches into database. If one of the batch fail then whole file should fail. Could you make a video on how this can be achieved? If i have to write it then all I could think of is to add a method to delete currennt file's records in exception block. May be this is a silly question but any response will be appreciated.

[@DefogTech]

That's interesting, let me think about it (how to implement it)

[@varunbhardwaj2267]

For scatter gather pattern can we try with for ExecutorService invokeAll() api because it will also return collection of futures and will retrieving using get() method we could timeout

[@Saurabhnaik]

Thanks

[@originaladmin]

Great vid. Although one doubt, this can be done using ScheduledExecutorService also, right?

[@DefogTech]

Yes, but still need synchronization/logic to understand when all tasks are done for each request

[@muesligangster]

wouldn't it be more efficient to use an event approach hier to fetch all the data? The computing time isn't that much and the overhead for threads could be much more than with an non blocken I/O such as NIO for example. And it would be much easier for scaling

[@saurabharaiyer]

Thanks!

[@nithishreddy9056]

Sir,i m very new to IT sector and from. Elsctrical background.. Please provide github link for accessing your code to my machine

[@divyangshah99]

Awesome

[@ashishpatil8066]

Nice video.. 1 query--> What if 2 threads completed and control returned to main method , but still thread-2 will keep on executing?

[@DefogTech]

Yes, if there is a timeout.. the threads will not know the main thread has moved on.. they will execute and complete their tasks, buy by that time main thread has moved on

[@ashishpatil8066]

@@DefogTech ok.. Thanks.

[@karthikmucheli7930]

Why not use thread.join(); ?

[@cihanisler6643]

This doesn't look quite efficient to me. What if there are 50 websites you need to visit ? Are you going to create 50 threads ? This is a web app as you specified and main thread is blocking up to 3 sec while waiting for other threads to complete their tasks, what if you get 100 requests at the same time ? Throughput will be very low and app will not respond. Also in terms of Cpu and memory usage creating one thread per website will not be efficient. I think the efficient solution is to use reactor java (non-blocking io) and event-loop architecture instead of thread-per request model for this kind of problem, especially if this is web app.

[@DefogTech]

You are right, if the use-case changes where 50 sites are to be hit and request throughput is high then either Reactive IO or a light-weight threads like Fibers (I have created videos for both of these topics) is more optimal. This video though is focussed on interview setting to understand candidate's knowledge about basic concurrency tools.

[@karimiumar]

Thanks!

[@DefogTech]

Thank you for the support!