Delta Tables

[@yurahulivatyi7955]

Great video. Thank you!

[@TheyCalledMeT]

did that really take 39sec to process 5 rows? why?

[@pauljeffcott8770]

The purpose of this type of tech is to work on big data, not tiny data (5 rows). Its distributed compute, so multiple computers splitting up the processing. The 39 seconds is the overhead to orchestrate and execute the request across the various systems. Seems like a long time when you just want 5 rows of data, but would seem short when you're pulling billions of records and seeing SQL jobs run for hours to days. Imagine turning on 5 computers to have each pull 1 row, kind of what's going on in that example. Just showing that it works, not that its the solution for the task at hand (reading 5 rows).

[@TheyCalledMeT]

@@pauljeffcott8770 I'm perfectly aware of overhead, just was surprised it's that much. It disqualifies the tech for several applications, which is what I was looking for But thx for the reply!

[@xa6859]

@@pauljeffcott8770 In other words: 5 rows will take 39 seconds, 500.000 rows will take 41 seconds, 5.000.000 rows will take 50 seconds etc. because the overhead is relatively big having small data but relatively small having big data.