I went through all the problems you mention.. When I first started using Hadoop / Spark / Flink I was very frustrating about all the very low level aspects you were required to master before being able to read or write any single piece of data I had the feeling of being the only one asking for data portabiliy and security. Having a common format for describing input/otput data (and metadata as well) is the fundamental point of any big or small data solution
@npestrov2 жыл бұрын
A really great talk! Looking forward to filling the box between storage and compute layers over the next few years.