Thread Pools in Java

A Thread Pool is an alternative to creating a new thread per task to execute. Instead, a number of pre-created threads exist in a pool - into which you can submit tasks to be executed by these pooled threads. New tasks submitted to the pool are first stored internally in a queue. From this queue the pooled threads will take the tasks and execute them. Each thread in the thread pool executes in a loop that takes a new task from the internal task queue in the thread pool (or block if no tasks are available), executes the task, and then tries to take another task etc. repeatedly. In this thread pool tutorial I explain in more detail how thread pools work, and show a thread pool implementation in Java. To access the code, click on the link to the textual version of this thread pool tutorial.
Chapters:
0:00 Thread Pool Introduction
1:49 Thread Pool Load Balancing Between Threads
3:00 Using the implemented Thread Pool - ThreadPool
6:48 The Implementation of the ThreadPool class.
12:10 The Implementation of the ThreadPoolRunnable class.
16:12 Summary of use of the ThreadPool implementation.
Thread Pool Tutorial - text:
tutorials.jenkov.com/java-conc...
Java ExecutorService Tutorial (built-in Java Thread Pool) - text:
tutorials.jenkov.com/java-util...
Java Concurrency Playlist:
• Java Concurrency and M...

[@hiendinh3871]

This Java Concurrency and Multithreading playlist gonna be the favorite playlist of my nephews, my nieces and my children when they engage themself in Computer Science one day!

[@JakobJenkov]

... if the world hasn't changed radically by then !! :-D

[@computerlearningbyargusaca5217]

Exactly what I was searching for .. extremely helpful

[@JakobJenkov]

Great to hear :-)

[@shubhamagarwal1434]

God of Concurency..i have bene flowing you since my 2014 when you used to write blog post only...by going through your post i attend interview like a LION when they ask mutithreading qns..Thanks a Lot form 10+ yrs exp guy from BLR,India.

[@JakobJenkov]

You are welcome !! ... I am happy that my tutorials were helpful to you over the years!! :-)

[@roman_mf]

Thank you Jakob for making multithreading fun to learn!

[@JakobJenkov]

You are welcome ! Great to hear :-)

[@user-sh1ep9qx3z]

this is one of the most understandable explanation and easiest to understand video on the topic

[@JakobJenkov]

I am glad you think so ! :-)

[@TimurLee-on7sm]

your explanation is so clear and the most important thing you show how it all works internally, thank you so much

[@JakobJenkov]

You are welcome! Glad you liked it! :-) ... seeing how a component works internally demystifies it - and enhances your general knowledge level :-)

[@shankar7435]

For a moment I thought ThreadPool was a Java standard class and wondered how I could miss this? Thank God. It's a custom one made by the author. No need to remember that. This should have been made it clear in the beginning.

[@chadchad4908]

thank you sir! you really helped me alot . you surely deserve more attention !

[@JakobJenkov]

Thank you very much! :-)

[@michaelfarhat4881]

With this great and simple explanation I hope that you can make a tutorial about data structures in java.

[@JakobJenkov]

I might - but I am not sure I will have the time for that... but I might write textual tutorials about this topic. That might still be helpful for you... but I am no there yet.

[@grankoczsk]

Just what I was looking for

[@JakobJenkov]

Great! :-)

[@menchomenchev8973]

outstanding content Jakob!

[@JakobJenkov]

Thank you very much :-)

[@alexa.2112]

Thanks a lot for your explanation. You've done a great job. For those java begginers as I who wants to try this code code on their machines I will suggest to do simple pause on PoolThreadRunnable class public void run(){ this.thread = Thread.currentThread(); while(!isStopped()){ try{ Thread.sleep(1000); //do simple pause there Runnable runnable = (Runnable) taskQueue.take(); runnable.run(); } catch(Exception e){ //log or otherwise report exception, //but keep pool thread alive. } } } Then you will clearly saw that the all tasks evenly distributed by the number of threads. Because in my machine in 80% cases all work done on Thread-0.

[@JakobJenkov]

You are welcome! :-)

[@tomsowyer4956]

Finally I got it )) Thanks

[@JakobJenkov]

Great! :-)

[@OODevelopers]

Thanks a lot to you. This multithreading tutorial is great. I understood a lot from this series. Can you please make videos on callable interface, future, future task classes even. Thanks in advance.

[@JakobJenkov]

Hi Siddireddy, thank you very much! I am happy you found this series useful ! ... yes, I will probably get into more depth about how to use more of the Java Concurrency interfaces and classes in future videos.

[@ThePomelo09]

Thank you!!

[@JakobJenkov]

You are welcome :-)

[@MohammedMutwakil]

Thank you very much

[@JakobJenkov]

You are welcome :-)

[@andymejia545]

@JakobJenkov is there any difference between ExecutorService pattern and ThreadPool and which cases are those can be suitables for complexity execution task?

[@JakobJenkov]

An ExecutorService is typically implemented using a thread pool of some sorts. No big difference. ExecutorService tends to have more functionality and more variants - that is all. Plus it is standard in Java.

[@kafychannel]

Hello Jakob, thank you for your great tutorials ! 11:26 Are you sure we should make execute to be synchronized? Can we just make stopped variable to be volatile ? I guess queue.offer() is already synchronized this is why we may not need execute() to be synchronized

[@JakobJenkov]

It is possible that you could optimize the synchronization around some of the constructs in this video. I try to keep them as simple as possible to avoid confusing the viewer, meaning sometimes a construct might not be optimally implemented - if that optimization is not the purpose of the video.

[@ayushsatyavarpu6055]

Hello Jakob! Thanks for the great video, I had a question regarding your implementation of the ThreadPool. I read over the code, and I am curious as to how race conditions are prevented here. Based on the code, could the case of two PoolThreadRunnable's both finish their run functions, then try to take the next runnable from the List? The only way I can see to prevent this is if the BlockingQueue's take method is synchronized, which I'm not sure if it is?

[@JakobJenkov]

BlockingQueue is a concurrent data structure, so only one thread can take an element at a time.

[@candemirhan01]

Hello sir, Thank you. That video series is what i was exatly looking for.

[@JakobJenkov]

Great! :-)

[@bahadirgirtten]

Thanks for the example and explanation. Do you think there would be a better way to implement waitUntilAllTasksFinished ? The current implementation would interrupt the last task if it were a long one. i.e TaskQueue being empty does not mean the very last task has finished its job.

[@JakobJenkov]

Yes - if you wanted to implement a ThreadPool 100% robustly - you would probably make a few changes to it. However, if I did, the video would get pretty long, and the code long to show on the screen, so I left it as an exercise for the viewer to do ;-)

[@akhlaq87]

When would you use your own implantation or your approach instead of the ExecutorService from std lib?

[@JakobJenkov]

Only in a situation where the ExecutorService lacks some functionality that you need. Otherwise there is no big reason to implement your own thread pool. However, I have come across situations where I needed something slightly different, and in those situations it can be handy to know how to implement your own Thread Pool :-)

[@akhlaq87]

@@JakobJenkov thanks for the reply and the great content

[@AlouiMohamedhabib]

NIce thank you.

[@JakobJenkov]

You are welcome :-)

[@shortsum]

is it possible that a thread can be preempted when it's inside it's critical section ? If yes what would happen in case the other thread is context switched, as this new thread would be unable to attain the lock as it's held by the preempted thread. All I am aware is if the executing thread is preempted before calling wait(), it doesn't release the lock.

[@JakobJenkov]

In theory, a thread could be preempted while inside a synchronized block, or while holding a Lock. However, no other thread can enter the synchronized block or take the Lock until the preempted thread wakes up again, and exits the synchronized block, or unlocks the Lock. When a thread is inside a synchronized block and calls .wait() on the object the synchronized block is synchronized on (the monitor object), then the thread is preempted and releases the "lock" on the monitor object, so other threads can enter blocks synchronized on the same monitor object.

[@shelendrasharma9680]

Hi Jakob, an amazing lecture:). I AM a big admirer of your knowledge. I have a doubt. why do we synchronize the below method in thread pool class as the blocking queue is already thread-safe ? why an explicit sync is required here? public synchronized void execute(Runnable task) throws Exception{ }

[@JakobJenkov]

You are right - it should be enough to just rely on the BlockingQueue to be thread-safe. I think I just synchronized the methods to avoid having to answer the reverse question: "Why is the execute() method NOT synchronized" ? :-) .... and thank you for your kind words! I am happy my videos have helped you ! :-)

[@SuperSam4y0u]

In PoolRunnable the Boolean variable is not volatile, will this not cause visibility problems? I saw that synchronised is there bt that is not enough rite? Pls let me know.

[@SuperSam4y0u]

The volatile concept has me thinking, if you have non final variables in a class and that class will be used by multiple threads then you should make it volatile. In addition if multiple threads are also changing(as in the reference) that variable, then make the write method with synchronisation.

[@SuperSam4y0u]

This explained it tutorials.jenkov.com/java-concurrency/java-happens-before-guarantee.html... i guess thanks

[@JakobJenkov]

If a variable is accessed from withing a synchronized block, then the variable does not need to be volatile, as synchronized blocks in Java have about the same visibility guarantees as a volatile variable.

[@zoltanzsurkai7725]

Hey Jacob. Why the tasks (that you add in the execute method in the main method) are Runnable implementations when you only call the run() method on those objects from the PoolThreadRunnable.run() method? So if I see correctly those Task objects could be any other (not Runnable) objects...right?

[@JakobJenkov]

Yes - in theory the PoolThreadRunnable.run() method could call _any_ method in the task objects - if you changed the code to do so. I just used the run() method - because developers are used to the run() method of such tasks :-)

[@zoltanzsurkai7725]

@@JakobJenkov Thanks for the confirmation!

[@abenezertilahun610]

Thank you

[@JakobJenkov]

You're welcome :-)

[@druzzyaka]

Jacob, as always, thank you for the lesson 🙂 I copied your code and slightly modified it to measure the execution time. Then I increased the number of tasks to 5_000_000 and ran the program with different amount of threads available in the thread pool. To my surprise, single threaded result is always 50% faster then multithreaded, although my cpu is dual core with 4 threads. Then I looked in the CPU utilization chart and found out, that even if I run the program in single thread, CPU load is distributed evenly between all 4 threads, ~90% utilization for each. Could you please explain such a counterintuitive behaviour?

[@JakobJenkov]

Hmm... it is possible that the 4 threads lose some time while being blocked - trying to take a task from the internal task queue in the thread pool. A single thread will not have that problem. However, if your tasks were downloading files from the Internet, I am sure you would see that the multithreaded version would perform better than the singlethreaded version! When downloading - the threads will be blocked most of the time waiting for a result from the remote server, so congestion on the internal task queue would be lower.

[@druzzyaka]

@@JakobJenkov Thank you for the answer! Is it possible, that the task for each thread is too easy in the example? So the frequency of queries to the task queue is too high for multithreading to be effective?

[@emmanuelreuven7350]

You are the best man, thank you

[@JakobJenkov]

You are welcome :-)

[@minasalah8210]

thank you.

[@JakobJenkov]

You're welcome!

[@azatakhunov6061]

good video!

[@JakobJenkov]

Thanks :-)

[@fuadgafarov]

thanks Jakob

[@JakobJenkov]

You are most welcome! :-)

[@shortsum]

What would the equivalent code using reentrant lock. Do we need to guard the take method as well ?

[@JakobJenkov]

That is for you to figure out :-) The more you learn about how concurrency works in Java, the easier it will be for you to figure out!

[@shortsum]

@@JakobJenkov Thanks, i tried multiple times and finally I understood what a lock and it's condition exactly does and how to effectively use it. I was able to get it work. In my opinion modifying your tutorials with lock and condition is probably the best way to start with. Also do you know of any resources where I can try out fixing the unsynchronized code and test out my knowledge. Thanks once again.

[@RitaTheCuteFox]

thanks

[@JakobJenkov]

Welcome :-)

[@mok8957]

thx for that

[@JakobJenkov]

You are welcome! :-)

[@IgorDomshchikov]

@JakobJenkov, thanks for the set of videos. Can you please provide more details about why the PoolThreadRunnable.doStop() and PoolThreadRunnable.isStopped() are synchronized? I am not sure I understand the purpose of this, as both are called from the synchronized methods of the ThreadPool instance.

[@JakobJenkov]

That is to ensure thread visibility of the member variables set by these methods, for the threads running the Runnable instances.

[@IgorDomshchikov]

@@JakobJenkov , thanks. The point was about the visibility. As I understand making isStopped variable volatile can be an alternative.

[@RaghuVN]

Great content, but I think it will be better if you do the implementation of threadpool before using it.

[@JakobJenkov]

You can always check out the description below the video and find a link to a textual version of this tutorial - then you can jump around the text as it fits your learning style :-)

[@alibaba40thvs]

Please also create some lessons on ThreadGroup.

[@JakobJenkov]

Hmm.... I might do that ! Thank you for the suggestion!

[@basavaraj2065]

Do we need ThreadPool when Executor service API available ? I believe Executor API are improvised version of ThreadPool

[@JakobJenkov]

If the ExecutorService provides all the functionality you need, then yes, use the ExecutorService, so you don't have to implement a thread pool yourself, and you don't mess up anything :-) Only if yo uneed some functionality that the ExecutorService does not provide, does it make sense to implement your own thread pool. Or - as a learning experience.

[@prometheusli101]

but the (max number of task) was inoperative,if i change the for loop number i to

[@JakobJenkov]

True - in the implementation shown in the video, if you add more tasks to the thread pool than it has capacity for, the extra tasks are just ignored. You can change that implementation if you need smarter behaviour :-)

[@redcrafterlppa303]

Great video. Except for the busy waiting in the thread pool. I guess you used it for simplicity but you really shouldn't busy wait.

[@JakobJenkov]

"...but you really shouldn't busy wait." . I disagree. There are situations - especially in low latency programming - where busy waits make sense because the overhead of waking up a waiting thread is high. Also, if the load on the system is high, the busy waits typically won't last very long, and thus won't result in a huge overhead. You have to know what you design for - when designing high throughput, low latency, highly concurrent systems. And you have to measure performance a lot! The same designs may even perform differently on different types of hardware. There is no "one size fits all" in this area. In this case though, I did use busy wait for simplicity. The alternative would have been implementing some more complicated thread signaling mechanism where the PoolRunnable's signaled to the ThreadPool when they had finished their work. The ThreadPool could then check if there were more tasks left in its queue (size() > 0), and if not - notify the waiting thread that it could wake up. That is probably how you would do it if you were to implement a ThreadPool for an API. But here I was mostly showing a way to implement a thread pool - not "The Perfect ThreadPool implementation".

[@user-un7tf9oo9r]

Hi Jacob, could you give us an implementation example of a threadPool using wait() and notify() instead of sleep(1). Thanks.

[@JakobJenkov]

Hmm.... at some point I might ... but if you are just looking to use a ThreadPool - the built-in Java ExecutorService is probably a better choice than making your own!

[@user-un7tf9oo9r]

@@JakobJenkov You are right, my software requires lots of evolutions at different points. for the moment a java executor does the job. I will need to create my own version but this can wait. I have many other parts of my software to make evolve before to come back on a better way to handle my thread pools. Th4n|

[@imr_07]

First, thanks for all the in depth tutorials. When I'm submitting i < 100 tasks, why am not seeing 100 logged messages?

[@JakobJenkov]

Does your code wait for the tasks to complete before exiting ?

[@imr_07]

@@JakobJenkov Yes, I call threadPool.waitUntilAllTaskFinished(); public synchronized void waitUntilAllTaskFinished() { while (!queue.isEmpty()) { try { Thread.sleep(10); } catch (InterruptedException ex) { ex.printStackTrace(); } } } It's only working correctly when I'm giving queue capacity == no. of tasks.

[@imr_07]

Issue is with this.queue.offer(task); inside execute() method. We need to wait before there is some space in the queue for adding more items. or using overloaded this.queue.offer(task, 100, TimeUnit.SECONDS); let me know if you think there is some other issue.

[@user-ry4ph5sj2p]

why waitUntilAllTasksFinished() needs to sleep for 1 ms if taskQueue size is > 0?

[@JakobJenkov]

It sleeps in a loop, so it keeps checking if the taskQueue size has changed - but since it may take some time for task queue size to change it does not just immediately check again. It sleeps 1 ms to give the pool threads a chance to finish more work. This is a simplified way of waiting. In a real implementation you might do it differently - using thread signaling (wait() + notify()).

[@aamirngr]

why execute function defination is declared synchronized when only main thread calls it? there is no other thread except main thread that calls execute method

[@JakobJenkov]

In case more than one thread was to submit tasks to the same thread pool, having its execute() method be synchronized is handy.

[@shortsum]

@@JakobJenkov Will it affect the synchronization , given the queue is already guarding it with lock in enqueue and dequeue methods. I tried this out and got an expected result.

[@animanaut]

i wonder if this is how in part an actor model is implemented

[@JakobJenkov]

Probably similarly, yes 😊

[@devdesk]

Code is not working with 100 runnable tasks.working fine with 10 runnable tasks

[@kafychannel]

Hello ! I guess it is because Jakob uses offer() method - this.taskQueue.offer(task); instead of put() in his example. Offer method returns true if it is possible to put a task into the queue and false if not. Because Jakob creates thread pool with capacity of 10 tasks it works when he calls threadPool.execute() 10 times. If you increase count of execution inside for loop you also should increase capacity of the queue or use put() method. put() - When the queue is full threads will be waiting for space to become available and then put their tasks into the queue

[@matimozg]

When I start main class first time I got exception and only 9 tasks done. I tried run main method many times after that but always program return without errors.

[@JakobJenkov]

Strange... if you find out why - let me know!

[@relax7105]

@@JakobJenkov I tried to investigate it with adding sleep of 3 sec in each runnabletask that is to be executed. The last task doesnot complete in this case. because in waitUntilAllTasksFinished method you checked the taskQueue size and based on that you waited for 1 ms and exited and called stop in main thread which stops all PoolRunnablethread. but even runnable.run task was sleeping for 3sec it got call to interrrupt and that's why last task couldn't complete.

[@relax7105]

@JakobJenkov I think this can be handled by increasing the thread.sleep() time in waitUntilAllTasksFinished() method.

[@mannysingh6618]

ThreadPool class does not need to synchronize at method level.

[@JakobJenkov]

I know... but that is a tiny difference in practice, unless you have many threads submitting tasks to the same thread pool. You can keep refining most components, but this video only serves ti illustrate the core principles... which it seems you already understand since you spotted the unnecessary synchronization.

[@anujcse06525]

If we put Thread.sleep(1000); below String message = Thread.currentThread().getName() + ": Task " + taskNo; there will be java.lang.InterruptedException: sleep interrupted

[@gregchen4023]

missing " " makes the code so difficult to understand, and you implemented 2 levels of "nested" runnable interface which makes the code with this mistake even harder to understand. I spent hours to understand the code in your text version of tutorial, till I found your video where you correct the mistake in the code. But your text version still has this mistake un-corrected.

[@JakobJenkov]

What missing are you talking about? That the BlockingQueue variable inside ThreadPoolRunnable is not declared as BlockingQueue ? You should be able to abstract from that pretty easily. The taskQueue variable is only referenced in 3 places, and one of them casts the obtained object to a Runnable. Also, the code in the video is the same that is copied into the article.

[@vinit720]

It is not working in hibernate application

[@JakobJenkov]

What do you mean "it is not working... " ? You need to provide more details if anyone are to be able to help you.

[@vinit720]

@@JakobJenkov Hi Jakob. Hope you would be well and doing great. My concern is that when I am running your code is running perfectly in standalone Java project main() method but when I am trying to run it inside a method in hibernate application multiple threads are not being created by thread pool. Hope u get what I trying to explain!

[@thesoftwareengineer17]

plz share the github link for same

[@JakobJenkov]

It's here : github.com/jjenkov/java-examples

[@gregchen4023]

the implementation is wrong, if the number of tasks is greater than the size of the block queue, it does not block and the tasks that are not in the queue are abandoned.

[@JakobJenkov]

That is how the current ThreadPool implementation is designed to work. If you want to block until there is space in the BlockingQueue, call it's put() method instead of offer(), or change the ThreadPool.execute() method to return the value returned from BlockingQueue.offer() . Simple. Done.