Building real-time data products at LinkedIn with Apache Samza

[@m1169199]

I like your slides, what did you use to make them?

[@CoderCoronet]

Hello Martin! Thank you very much for sharing such valuable content. I’m trying to find your video about building robust data infrastructure with logs. The link to the video on the talk transcript is broken. Can you share a new link to that video? Thank you!

[@MrMukulj]

Great talk Martin. Very well done!

[@pinhusdash6895]

What happens when a user from one partition views a user from another partition. How does the enrichment happen? Do you send a copy of the event to both partitions?

[@pinhusdash6895]

Pinhus Dash I think you kind of answer this at 46 minutes. But it does seem to double the time.

[@SamBessalah]

Great talk Martin.

[@pollathajeeva23]

TimeSeries tool may be more than

[@dudeabideth4428]

Isn't that a big database of profiles to have a copy? Or is it only the subset we care about? It sounded like the profiles replica got created from every profile edit event. So it sounds like a full replica

[@tomhpolo]

I might be wrong, but in his talk it sounded like there are 2 levels of partioning: by user and by job. By job: The stream processor for PageViewEventWithViewerProfile doesn't need all data from the EditUserProfile event, so it grab/replicate whatever fields it wants from that event. By user: If you partition users into different processors (ie: profile['id'] modulo N), then each replica only has that % of users in it.

[@houssemghazala]

👏👏👏🙏🙏🙏

[@GlebWritesCode]

I would say this talk has very little to do with Samza. Just a general view how LinkedIn does stream processing