Teaching old Streams new tricks By Viktor Klang

[@dazraf]

Love Viktor's presentation style and content. Learnt loads from this presentation. It's great news to see so much progress in the JDK!

[@viktor_klang]

I really appreciate that, @dazraf-thank you!

[@admendoz25]

I am only 12 minutes into this presentation and I've learned a lot. Great work Viktor

[@viktor_klang]

Thank you so much!

[@b_two]

Love the inclusion of an andThen api!

[@SwarangaSarma]

Amazing work! Showed me operations that I didn't even think I needed but I do now.

[@viktor_klang]

That's awesome to hear!

[@TheodoreRavindranath]

Amazing work. Looks a lot inspired by Reactive frameworks. But, this is much simpler and sounds like this would be a great addition to Java.

[@lfeng0766]

Amazing presentation.

[@viktor_klang]

Thank you!

[@hepin1989]

Nice and detail explain

[@viktor_klang]

Thank you, @hepin1989!

[@Muescha]

I expected the Gatherers to include all the methods from the java-util-stream-Stream as well (perhaps in a different class like `DefaultGatherers` or `CoreGatherers`). This way, I could also use these methods to create connecting Gatherers with the `.andThen` function.

[@Justjames283]

Great presentation!

[@viktor_klang]

Thank you! :)

[@Jankoekepannekoek]

I hope this will come as an incubator api soon.

[@bizoitz86]

Will Collectors do something about short#-circuiting as well, or will we have to short circuit in gatherers and then reduce? I.e. implementing allMatch.

[@viktor_klang]

I personally don't think that's possible to do as the evaluation strategy for Collector is already established and no current evaluation implementation checks for short-circuting. As such, I suspect that it would be hard to implement it in a backwards-compatible way.

[@IgnacioVaras-o3n]

Cool stuff, is there any way to play with it, other that implementing it myself?

[@iplantevin]

I hope this will make its way beyond previews, to catch up to, and to some extent, improve on Kotlin's more powerful collection API 🤞

[@avalagum7957]

When will we have these things in Java to play with? If Viktor Klang talks about this, I'd like to see the comparison with Scala.

[@viktor_klang]

Follow the progress of JEP-461: Stream Gatherers for more info

[@DesuTechHub]

I see this gather is somewhat similar to spark functions data frame api's expr functions. Happy that, we are taking good things...

[@Sakrosankt-Bierstube]

5:10 Ngl, i am a little bit conused.. reorder() exists as sort(), duplicateConsecutive() exists with distinct(), groupdUsing() exists through collect() and of course mapConcurrent() exists trhough map(). Yes.. you have to do some extra stuff instead of just putting a your lambda operation as argument but... it exists. All of that would be just another method name or syntax.

[@viktor_klang]

My thinking is that reorder() can emit elements as they are determined to be next in order (using sequence numbers or otherwise), whereas sort can only start to emit once the entire upstream has been consumed-this will never terminate for unbounded streams. deduplicateConsecutive() is definitely not the same as distinct() as you only want to eliminate adjacent duplicates, and collect() is a terminal operation and not an intermediate one. And, unfortunately, mapConcurrent() cannot be implemented using map() since you need to flush during finish, and the contract of map() is also that the supplied function is stateless and non-interfering. I hope that helps :)

[@DidierLoiseau]

I was kind of expecting at the end: “… and this is why we have decided to deprecate all intermediate operations in the Stream API in Java 23” 😅

[@viktor_klang]

😂

[@DidierLoiseau]

​@@viktor_klang Didn’t expect you to notice my comment! Since you are here, for the mapMulti() example at 26:07, shouldn’t the lambda return !downstream.isRejecting()? Also, I’m curious about isRejecting(), wouldn’t most usecases negate the returned boolean? Might be better to have isAccepting() instead - it would be more consistent with push().

[@viktor_klang]

⁠@@DidierLoiseauSorry, completely missed your reply. Having mapMulti return !downstream.isRejecting() wouldn’t hurt, but on the other hand it doesn’t really fix the problem that the user-supplied BiConsumer cannot know when it must stop pushing. As for isRejecting instead of isAccepting is that we do not know if it is accepting, we only know if it is rejecting. isRejecting() is primarily intended to be used when the upstream wants to avoid performing expensive work which cannot be passed downstream anyway. For instance in the finisher.

[@quickruss05]

Love this as I've run into many cases where I've wanted to extend the streams API. I even ended up building something allowing arbitrary operators based on mapMulti since it was the first official operator to provide access to the downstream sink but it had many shortcomings and didnt parallelize, etc. I kept expecting him to point to the library that has this but then disappointed this is not available yet 😢. Looking forward to this. Some feedback though... gather and gatherer don't seem like the right names, why not operate and operator? Also, where does this fit with reactive streams API? Java streams are great but I've found gaps like it being pull-based (and not push-pull with backpressure like reactive streams) and doesn't inherently support asynchronous operations that have made me use reactive streams APIs instead. It would be good to incorporate these concepts into Java streams. That with virtual threads would truly kill reactive streams.

[@viktor_klang]

Java Streams are push-based (in general) over a Spliterator. As for naming it is always a challenge-as you can imagine, a lot of different names have been suggested (would be a long conversation to fully elaborate all the concerns to address). What gather+gatherer had going for it is its relationship with collect+collector :)

[@USONOFAV]

So... Gatherer is in Java 21 yet?

[@viktor_klang]

No, it is not in Java 21

[@zaccicala2341]

Maybe preview in 22

[@DidierLoiseau]

@@zaccicala2341 indeed it is!

Looks like most of this featurers Reactor already supports

[@Muescha]

I would anticipate having both `Gatherer::endThen` and an alias `Gatherer::gather = Gatherer::endThen`. Why, you ask? This would enable me to effortlessly extract a group of Gatherers from an existing Stream pipeline. Otherwise, I would have to go through the process of renaming them step by step, changing from `.gather()` to `.endThen`, while retaining the same functionality.

[@viktor_klang]

I decided against doing so because I found that it made code reviews more difficult to understand (am I looking at Stream composition or Gatherer composition here?). Reusing the name for composition used by things like Function made more sense to me.