If I have a app ingesting CSV data into Kafka, can I use avro schema for that CSV data ?
@axual3573 Жыл бұрын
Yes, that is very possible There are two ways to do this. 1. A full, specific record definition The full record definition will have a namespace and record name reflecting the specific data you want to store. Each column of the CSV will become a named field inside the record. This field must have the correct type of course. Then you alter your application to map each CSV record to the correct Avro record type. 2. A generic CSV record definition If your CSV data is more flexible, for example, each CSV can/will have different headers and value types. This makes it harder to specify a specific Avro record definition. But you can make a specific Avro record and name it something like CsvRecord. It will contain only one field which is an Avro Map. The value of that Avro map can be a union between null, string, int, float, etc. This allows you to store each column in the csv as a separate entry in that map. You can read up on the Avro specification and how to define a schema on avro.apache.org/docs/1.11.1/specification/ Hope this answers your question.
@nishikanttayade7446 Жыл бұрын
@@axual3573 Thanks for taking time to reply 👍
@axual3573 Жыл бұрын
@@nishikanttayade7446 if you would like to explore Axual's event streaming platform, you can request a free trial right here: axual.com/trial/ :D
@calorus2 жыл бұрын
Excellent video. I've just watched through, will need another few watches to pick it all up !
@pauldrage99312 жыл бұрын
This was really helpful - there's a few interesting operators out there now which take care of abstraction - but knowing what's going on 'under the covers' as per this talk is really helpful when things don't go right!
@Vlad-zy4sl2 жыл бұрын
One of the most useful and concise explanations about Kafka in Kubernetes that I’ve come across. Congrats
@PeterPeerdeman4 жыл бұрын
Wow team axual great work! Both on the release and the video production quality. Awesome to see the stream message browser in action!