Understanding how ForkJoinPool works

ForkJoinPool is an advanced version of ThreadPoolExecutor with concepts like work-stealing which enable faster and efficient solving of divide and conquer algorithms.
Channel
----------------------------------
Master difficult programming concepts in few minutes. I try to explain difficult concepts like Java concurrency in simple to understand manner using animations and small code snippets. Explore videos on topics like Spring Boot, Cloud Foundry, Java 8 (with more coming soon). I am happy to clarify your doubts. Ask me anything in the comments. I am also happy to take requests for new videos.
New video added every Sunday.
Subscribe or explore the channel - bit.ly/defog_tech
Playlists
----------------------------------
Java Executor Service - bit.ly/exec_srvc
Java Concurrency - bit.ly/java_crncy
Spring Boot 2.0 - bit.ly/spr_boot2
Java 8 - bit.ly/java_8-11
Intellij IDEA Shortcuts - bit.ly/i_idea
Popular Videos
----------------------------------
Executor Service - • Java ExecutorService -...
Introduction to CompletableFuture - • Introduction to Comple...
Understand how ForkJoinPool works - • Understanding how Fork...
Java Memory Model in 10 minutes - • Java Memory Model in 1...
Volatile vs Atomic - • Using volatile vs Atom...
What is Spring Webflux - • What is Spring Webflux...

[@kimsung2384]

Bro - that was a cool video. I also watched your ExecutorService one which was great!

[@DefogTech]

Thanks buddy

[@karandua9848]

Watched your ExecutorService series, now came to this one. Really needed this set of videos. Nobody could have explained these concepts in so less time! SAVIOUR.

[@kalyanbhadra3202]

In the last many years, I was trying to understand these topics and never understand them fully, unless used some of them in projects. Your videos are awesome. You made videos with topics which are hardest to understand and in a so simple way that I never found before. Thanks a lot. You saved lot of us.

[@RaviPatel-po2gu]

Great Videos is it possible to cover the life cycle of Thread and its methods, how it works, ExecutorSerivice -> how it works internally, how it will reuse the same threads, in the prospective of the code

[@aaravpilligundla5270]

Complex concept(MULTI THREADING) is explained in much simpler way, where an average . like me can also easily understand all the multithreading concepts...If anyone wants to master multithreading, these videos will help a lot..Great job bro..

[@SuperDeb1985]

hi one question abt ExecutorService and ForkJoinPool . for e.g i am submitting one task to CompletableFuture.runAsAsynch(taskA, forkJoinPool) then forkjoin pool will do the work of forking taskA?

[@DurgaShiva7574]

so which is better, executor-service or FORK-JOIN pool, can both be used inter-replacably (i know will have overheads), and most important, when to use what, and why ??? if anyone can please guide, will be very helpful, thanks in advance.

[@himaninegi93]

I have one scenario for which i need clarification. Consider my task is getting a user Info. now inside this parent task, i have 3 diff calls; 1 call to an 3rd part API which collects the user's recent transactions, 1 call to DB that gets user basic info(name, manager name, address etc) and 3rd call to get its profile picture. All 3 of these calls are independent and can run simultaneously and in the end, we just need to combine the data and send it back. Since you mention that forkjoinpool should be avoided in case of any IO operations, how do we manage it if I don't want to assign threads to these sub tasks individually?

[@jineshsanghvi1]

I have 2 questions - 1. One of the thread (from the thread pool) pick the main task from the common queue and breaks it into subtasks and stores on its own local Deque. It then process each subtask in squence. In that case there is no parallelism involved but there is over head to break the main task into subtasks and then later joining the results (when no work stealing involved). This will be an over head. Aren't we better off just to execute the main task in whole ? 2. In the earlier part of the video, you mentioned that the main task is split into subtasks and these subtasks will run in parallel on different cores. In that case where is the local Deque involved ? Or is that the sub tasks will run in parallel on when other thread has no other main task to executed (no other task in the common queue)... Work Stealing. Overall , its a good vidoe and i have gone through all you other videos. Easy to understand both with language and graphics. Commdendable job!

[@batmanish]

As far as I understand, you don't want to execute the main task as a whole, since you can perform fib(n-1) and fib(n-2) parallely. There are 2 cases: 1. Say all threads are busy, if you use ForkJoinPool, even though you don't get parallelism, you still can still leverage data locality benifits, if you don't use forkjoinpool the subtasks might go to different Thread and you don't benifit from data locality. In this case since all threads are busy, there was no way to leverage parallelism so atleast we were able to leverage data locality. 2. Say some threads are idle, and thread1 is busy and has lots of subtasks in it's local queue, other threads can take up these tasks(stealing) from local queue and leverage parallelism (We loose up on data locality but that's okay we get parallelism). So in both these cases we were able to get some optimization (data locality or parallelism). Thus using ForkJoinPool makes sense.

[@_abhishekmj_]

Is forkjoinpool better than recursion? Otherwise we already have recursion right?

[@bharathashok9965]

Please provide a working programming example of ForJoinPool. Thanks for sharing knowledge.

[@narayanadhurti1603]

Splendid and Explanation with an intoxicating voice.

[@criticalthinking123]

Thank you to explain this simply and clearly.

[@chanderprakash4343]

Seriously i really appreciate your work and the way you simplified all these concepts.

[@mailtosmrutiranjan]

It would be good , if you can add performance metrices , means which implementation will give better performance , i know all are different threads calling same function , but still the internal implementation of CompletableFuture /CountdownLatch must have their own advantages.

[@RohitSharma-ke1bq]

Merge sort would have been good example for Fork Join. Nice work by the way. Awesome content.

[@stoical59]

Is there a email Id where we can post our problems?

[@HowISeeThat]

Sir please make new videos

[@gurucharansharma5574]

How does ExecutorService behave in the case of the task -> sub-task scenario? How does ExecutorService handle sub-tasks?

[@hyperborean72]

Would you say that ForkJoinPool is identical to some other more tradional pool (ExecutorService) combined with CompletableFuture?

[@Knigh7z]

When you say “task” and “sub task”, I presume these include general callables and runnables too? Like let’s say we use the execute or submit functions - you’ve submitted a callable to the fork join pool, and if that callable spawns more callables, are they considered sub tasks as well? Or do we only get benefits if we use those fork join specific APIs and pass ForkJoinTask instances.

[@hyperborean72]

How does RecursiveTaks relates to CompletableFuture if they bear any relation?

[@waterflow4711]

what do I have to do to make the code at 10:25 to work? When I input 5 it returns result 5.. but it needs to be 7.. plz help

[@tayybaadien4900]

can anyone please tell me the purpose of .......in RecursiveTask.... 😢😢😢

[@aslam-kg4xo]

That is the object type that will be returned by the tasks. Take a look at the return type of *compute()* method. it is also Integer. hope it helps 😊

[@prabhusamad4527]

wonderful man.

[@chinthakadharmasiri]

awesome explanation.!!! nicely explained and really liked the fibonacci example..!!

[@manjuwadhwa8080]

Please make more such videos focusing interview topics...your videos are quick and to thenpoint..awesome, Thanks.

[@aadityaraj4691]

Very good explanation, I have one question, how threads are created in ForkJoinPool? Is it one thread for each task? Else any fixed number of tasks when pool constructed?

[@daelous88]

Really usefull, i can understand this, thank you men.

[@tulikamal]

Nicely explained, thank you very much

[@mykytapiskarov7291]

Thank you for the materials. Have may you try to use ForkJoinPool with Java7 to represent Java8 parallelStream

[@georgejnirappeal9657]

Could you please explain how sub tasks managed in executer service?

[@Lahiru_Udana]

Thank you. Clearly explained

[@divyangshah99]

Just to clarify, thread will wait for all sub tasks to complete when join method is called, so it won't take a new task until all sub tasks are over, correct?

[@BhawaniSingh-jc7cw]

Now I feel it is very simple ...Great...Thanks :)

[@blueattube]

superb analysis and content.. hats off!! superb!! Lovely present with pleasant colours. :)

[@meenaljain3828]

ExecutorService is the interface..in the hierarchy we further have ThreadPoolExecutor class and ForkJoinPool class..so both are executor service. Shouldn't there be comparison between ThreadPoolExecutor and ForkJoinPool ? or am i wrong somewhere?..However the video is awesome.

[@DefogTech]

You are right, in fact internally ForkjoinPool is used in CompletableFuture/Streams API. Its a better version of ExecutorService in some ways and is thus preferred

[@Mr.RaviTeja]

Supper....👌

[@rohitjain1447]

How can we avoid work stealing when using ForkJoinPool?

[@berndeckenfels]

Great intro. The return with task1.join()+task2.join() does not free the calling task, forcing all other subtasks to be stolen. Might be a good topic for optimizations video.

[@roudder6606]

perfect offtopic: was it kettle?)

[@DefogTech]

lol, yes it is :)

[@rishiraj2548]

Thanks

[@أحمدفلمار]

Thanks

[@NikosKatsikanis]

who else came here expecting rxjs?

[@dmytro_glory_ukraine]

Top!

[@subramanyamsattenapalli4720]

The way of explanation is simply superb. Any one can easily understand.

[@nikhilpawar7876]

Best explanation i have heard ever❤

[@VikiSangani]

Very nicely explained, please keep posting such videos. Thank you very much.

[@sanjoyganguly9721]

please share some more real time example to understand the implementation

[@jagabandhumallick9538]

Pretty clear explanation, thanks 🙏

[@filipfolkesson3865]

You are making this course too easy! Thanks a lot

[@dineshchandgr]

amazing video bro. thank u

[@pandit-jee-bihar]

Excellent video in a short duration!

[@pankajs495]

excellent sir . what a amazing video

[@vamshikrishna8143]

I very much like your way of delivery. could you please explain why "Do not perform Blocking IO operations" at 12:20 not recommended.

[@DefogTech]

Thank you! Blocking IO is not recommended because ForkJoinPool tries to use limited threads, running on same cores, to take advantage of data locality and is thus faster. If we block the thread because of IO, ForkJoinPool will have to assign a new thread and flush the cache, which will still work but will be slower. In fact, then it is similar to using ExecutorService in my opinion.

[@SosetaFurioasaJr]

Good explanation, thanks!

[@shellindebted5328]

Thanks a lot for this video.Very nice explanation of ForkJoinPool concept.

[@richardwang3438]

I like the way you hiccup. LOL

[@spaarks84]

Awesome video. Can you maybe demonstrate how you could use memioization to re-use already computed values across the various threads in order to optimize for speed. I know you said to avoid synchronization, but I would like to assume there is some elegant way.

[@DefogTech]

If we want to re-use calculated values and make it available for all threads we can use ConcurrentHashMap instance which is shared/global. Internally it uses segmented locks which ensures good performance.

[@hariiyengar]

Grt Job Ajay. Very helpful. Thanks a lot

[@DefogTech]

You are most welcome! I am not Ajay. Having said that, he is a friend and I too, am a fan of his work :)

[@shikharchoudhary7639]

I java 8 parallel stream also uses fork join pool internally. But i dont know how it creates sub tasks.

[@DefogTech]

ForkJoinPool need not always fork tasks, it's also used as a work stealing thread pool

[@pradeepanbaluchamy8469]

Can u pls explain bitwise operations .. xor.

[@TUHINT]

can u plz add a git link for the demo program

[@mahendharkhedhar7887]

I'm big fan of u r teaching ,always eagerly waiting for u r vedios sir really ,we are very lucky to have you in this channel

[@DefogTech]

Thank you so much for the kind words and unwavering support! That's what keeps me motivated.

[@MadhuKumari-gy7uz]

Wow you're a saviour man🙌

[@RepublicDay2023]

Wao superb. Nicely explained. Thanks buddy.

[@DefogTech]

You're welcome sir!

[@vigneshbalachandran9704]

Thanks bro for sharing.

[@raghu3055]

simply superb bro! the way you explained is :) can you please make some videos on Java 8.

[@DefogTech]

Thank you sir! I definitely plan to add few more videos on Java 8

[@vijisdiary4902]

Can you post video on the difference between Synchronised method and Synchronised block

[@DefogTech]

There is no difference... Synchronized method is same as using synchronized (this) for whole method body.. Synchronized blocks are recommended when you want to have mutual exclusion for subset of a method or when you want to lock on some other object than 'this'.

[@jineshsanghvi1]

@@DefogTech The one main difference between synchronzied method and block is - In case of sync method the thread has to gain lock on the object of that method, where as with synchronized block you can provided any object whose lock that thread has to get to enter the critical section

[@manasranjan4]

Thanks a lot. Very nice and clear explanation

[@jetiax]

Thanks. Very clear.

[@josephmhlanga172]

Great stuff

[@mukeshsinghdance7332]

Lovely

[@roman8745]

that's great, thanks

[@sumanghosh4299]

thank you , it helped me a lot

[@aminhojomi4777]

ty