Watermarking in stream processing | Course in Spark Structured Streaming 3.0 | Lesson 7

[@nilanjansarkar100]

This is the best video on watermarking on KZbin! It's so practical, most literature is focussed on high-level mumbo jumbo except actually explaining how it is actually implemented. Kudos to you Amit for making this.

[@danieltost]

Man, your explanation is awesome. Thanks.

[@DataTechByAnandKumar]

One of the best explanation about the watermark. Thanks

[@izebit]

Amit, уou saved my day. Apache Flink has the same feature, however, the documentation about it is not clear. Thanks to you, eventually, I understand how it works.

[@nilanjansarkar100]

Man, I have been hunting through the flink documentation for so long, they have every other details except how Watermarks is actually calculated. Their docs is pathetic

[@williamson-mcgee7365]

Thanks for the video! This is really useful

[@vladvlad2798]

amazing explanation! thank you!

[@deepanshuaggarwal7042]

In "UPDATE" output mode, we get output after each trigger time unlike "APPEND" mode where we get data only at the end of watermark. I have one more doubt ralated to "UPDATE" mode, do we get our data again in the output (it can be console or kafka) when watermark time ends?

[@amit-ranjan]

Good question. It will be interesting to try it out, but my first thought is that with 'update' mode, you don't get it again after the watermark ends.

[@deepanshuaggarwal7042]

@amit-ranjan I also think so but not sure.

[@karanalang1573]

excellent video, thanks for the detailed explanation! Is the watermark implementation using Kafka as src also covered in a different video ?

[@ambar752]

As per your slide output, 12:17 should be Dog instead of Owl, please correct it...

[@kumarankit4479]

Sir i watched this video and really loved the way you have put. But i am also very sad to see most of the videos on your channel are unavailable. Can you do something for that?

[@amit-ranjan]

Thanks! I've updated the permissions on some of the videos. Can you check now?

[@vinothvk2711]

y time window starts from 11 55?

[@ambar752]

However, I liked the video, superb explanantion...