129. Databricks | Pyspark| Delta Lake: Deletion Vectors

[@sravankumar1767]

Superb explanation Raja 👌 👏 👍

[@rajasdataengineering7585]

Thank you so much 🙂

[@piyushagarwal79]

what is the difference between REORG & OPTIMIZE then

[@venkatasai4293]

Good video Raja . Could you please make a video on liquid clustering with example illustrating the difference with normal partitioning .

[@rajasdataengineering7585]

Hi Venkat, good suggestion. Sure will create a video on liquid clustering soon

[@venkatasai4293]

Thanks raja .

[@akashghadage5377]

Thanks!

[@rajasdataengineering7585]

Welcome!

[@RaviY-o6r]

After applying optimise command the behaviour is same with out enable deletion vector. What use case we use deletion vector.

[@rajasdataengineering7585]

Yes that's right. But we usually don't run optimize command frequently and its not recommended also. So the use cases are like where we need to manipulate the data frequently. This is going to big boost to performance and storage

[@venkatasai4293]

@@rajasdataengineering7585why it was not recommend to use optimize command ? Any overhead ?

[@rajasdataengineering7585]

Optimize command is rearranging the data files which is costlier operation. So it's not recommended to run so frequently. When we accumulate significant amount of data from previous run, we can run optimize command