Virtual Threads in Java 21

In this video we'll cover a relatively new feature to the Java language, VIRTUAL THREADS. We'll talk about some current BEST PRACTICES of using them and cover SIMPLE EXAMPLES of how you can use them in your code.
I'm Will Tollefson, a guy who loves to NEVER STOP LEARNING and teach others what I've learned. If you're enjoying the content, please LIKE, SUBSCRIBE, and SHARE!
LINKS:
JEP 444: openjdk.org/jeps/444
VIDEO CHAPTERS:
0:00 Introduction
0:49 JEP 444
2:14 Use Case Example
5:02 Virtual vs Platfrom Thread Performance
7:26 Throughput vs Latency
8:10 Common Interactions With Virtual Threads
11:38 Java 21 Current Best Practices

[@willtollefson]

What do you think about virtual threads? Do you plan to use them in your code?

[@blaaarp4u]

I have a great time watching your amazing vids! Greetings from Jerusalem and thanks again.

[@willtollefson]

Glad to hear it, thanks for the comment!

[@DenaTollefson]

Awesome tutorial, Will! I appreciate how you stay current on new releases and features 😀

[@willtollefson]

Thanks! I think virtual threads are a cool concept added to Java, but also have some hidden details that I feel like I'm still uncovering as I get more experience using them.

[@chamanjain9843]

I have a question. Suppose I made an HTTP request using Java from a virtual thread, let's say using RestTemplate. Now, suppose that the response takes approximately 2 minutes. Does this mean the virtual thread will release this platform thread and OS thread and itself get blocked for the duration of 2 minutes? Assuming the response arrives on port 8080, will it directly notify the virtual thread that the response has arrived, prompting it to wake up? Or will there be some OS thread that handles the response first and then informs the platform thread to wake up the virtual thread?

[@willtollefson]

If I'm understanding the question correctly, this is a perfect example for using a virtual thread. With the previous thread model, when you use a synchronous API like RestTemplate, the OS thread ends up in a blocking call waiting for a response. When your application process services other requests in this state, it would need to use other threads, sometimes creating new ones, which can get expensive when you scale up. With virtual threads, the virtual thread would be in the blocked waiting state, but you could start other virtual threads while servicing other requests and potentially have them use the same OS thread. This essentially limits the overhead of creating more OS threads than you need and it can help your application throughput in cases where you have virtual threads doing blocked waits, IO operations, and that type of thing. Hopefully this helps!

[@chamanjain9843]

@@willtollefson Thanks for explaining, but my question is: Actually, I don't understand if the virtual thread released the OS thread, then how it got notified about when to resume. Suppose I made 5 requests using 5 virtual threads, then all got blocked and associated OS threads got freed. Now, when we get a response back, how does the JVM know which virtual thread made which request? I am assuming that when we get our response to our Java process, some OS thread must consume that response from the specified port number on which our app is running. So, how do these OS threads know which virtual thread made which request?"

[@willtollefson]

@@chamanjain9843 yes when a networked connection like that happens, an OS thread would initially service the connection from the perspective of the JVM, but I might under the hood switch contexts to on of the thread’s virtual threads. The way it knows which virtual thread to map to is similar to how Java today knows which thread receives a response when something is sent back to a process. The response handling code will either kick off an OS thread to handle the response if it’s an asynchronous API or call back to the waiting thread if it’s synchronous. For virtual threads the process looks really similar. To a lot of the JVM, a virtual thread looks like a regular thread. Some of the web frameworks will likely need to be updated to kick off virtual threads to service requests or find call backs rather than using a thread pool like most today purely from the perspective of efficiency, but aside from some pinning concerns that haven’t been addressed yet by the language, this sort of detail should largely be handled by the JVM rather than application code

[@rodjenihm]

Virtual thread doesn't need to "notify" the platform thread once it resumes. The same virtual thread can run on different platform threads. Virtual thread can start doing the work, start some IO operation that blocks at which point the platform thread will be released. After the IO operation is finished and the virtual thread is ready to resume it will be scheduled on any platform thread available. Initial platform thread does not need to get notified. It can be the same platform thread but it does not have to be.

[@ymetelkin]

It takes 1.3 seconds on average to run 100,000 concurrent "1 second sleep" tasks in Go. This is 4 times faster than in your example. Is it expected? I didn't run Java version on my machine (I don't have it), but my laptop is not a super computer by any means. And just for the fun of it, I ran with 1M concurrent tasks and it took 3.1 seconds on average. And then, for even more fun, I ran it with 10M tasks; finally it was slower than 5 seconds, 11 seconds to be exact. I wonder if you can try your code with 1M tasks and share your results.

[@willtollefson]

Thanks for the comment! When I run 1M tasks on my machine with the Java version, I'm getting 10-11 seconds. When I run similar Go code on my machine I'm getting around 5 seconds. Without deep diving the compiled byte code and go binary to really figure out the performance difference, I have a few upfront guesses: 1. Java warm up could be playing a small role here due to lazy initialization for things like the classloader and virtual threads mapping to platform threads 2. Java and Go have different default behavior for the number of routines that can be executed in parallel and the number of OS threads available to the runtime for scheduling. 3. Goroutines are optimized to have a small stack size at the beginning of execution and then expand as needed. I'm wondering if stack allocation for Java virtual threads is more expensive in this case. Both are still MUCH better in this example than the previous Java thread model though :)

[@ymetelkin]

Sleeping for 1 second is probably not a very realistic example and goroutine smaller starting stack may explain the difference