Google SWE teaches systems design | EP40: Flink in 15 Minutes, Stateful Stream Processing!

[@ahnjmo]

Hey Jordan! Quick question - on your slide at 04:51, how come we would need apache flink versus a regular message queue to ingest changes from a stream? Or would they both be sufficient for this particular usecase?

[@jordanhasnolife5163]

I think that you're slightly misinterpreting the point of flink here. Flink isn't actually a message broker, but works with a message broker. It takes in messages from a message queue, and has local storage that is useful for performing computations on the messages that it has received in the past. The message broker is just responsible for getting messages from producers and delivering them to consumers.

[@vincentchee4108]

You explain it so well

[@raj_kundalia]

thank you!

[@shubhankar915]

How do we create checkpoints? I have seen checkpoints being used in all kind of cases. May be a new video idea for you?

[@jordanhasnolife5163]

Look up chandy Lamport checkpointing, may be a bit beyond the scope of the channel lol

[@vincentchee4108]

@@jordanhasnolife5163 Maybe you can cover it

[@SreekantShenoy]

oh no. i am dumb.

[@jordanhasnolife5163]

That's a lie!