114. Databricks | Pyspark| Performance Optimization: Re-order Columns in Delta Table

Azure Databricks Learning: Delta Lake: How to re-order columns of a delta table?
=================================================================================
Re-ordering tables columns is one of the most common requirement in database and data warehousing concepts. It is also improving performance in Databricks delta lake. To know more about it, watch this video
• 114. Databricks | Pysp...
#Deltalake #DataSkipping, #DeltaSkipping, #ReorderDeltaColumns, # RepositionDeltaColumns #SparkDevelopment,#DatabricksDevelopment, #DatabricksPyspark,#PysparkTips, #DatabricksTutorial, #AzureDatabricks, #Databricks, #Databricksforbeginners,#datascientists, #datasciencecommunity,#bigdataengineers,#machinelearningengineers
Delta Lake Internals: • 52. Databricks| Pyspar...
Z-ordering: • 66. Databricks | Pyspa...

[@SriRamTrading]

It's good and new for me as per my experience but here I think how it's make performance optimization

[@rajasdataengineering7585]

Pls watch the complete video and you will understand how it improves performance

[@SriRamTrading]

@@rajasdataengineering7585 in simple it just change the order of column, correct me if m wrong

[@passions9730]

Finally ur back..

[@georgesudi]

Hi, Thanks for sharing those features! It helped me a lot

[@rajasdataengineering7585]

Glad to hear that! Thanks for your comment

[@AshokKumar-ji3cs]

Welcome back brother

[@rajasdataengineering7585]

Thanks brother 👍🏻

[@prabhatgupta6415]

Cover real time scenarios in databricks sir. As well as cicd.

[@rajasdataengineering7585]

Sure, will do

[@teluguoldmemories8325]

For delta don't specified using delta it will take automatically you can check also using describe extended command.

[@sambitmohanty1758]

Hi can you please make a video on DE project end to end

[@rajasdataengineering7585]

Hi, I have already created couple of videos on this requirement. You can check if that helps you kzbin.info/www/bejne/mqnbiYqbacyBitE kzbin.info/www/bejne/f5KZl3eindF-jrM

[@vidyasagar6597]

Hi sir, Can you please create a vedio on why and when we opt for different scripting pyspark or scala or java or R.. what's the difference?

[@vineethreddy.s]

At 13:19 how are you able to append the data with the schema being different. Shouldn't we use mergeSchema?

[@pde.joggiri]

Hi sir FSCK vs MSCK please explain? We mostly face manual file delete error/ file not found error and FSCk resolve the issue When can we use MSCK

[@rajasdataengineering7585]

Hi, sure I will create a video on this requirement

[@bachannigam4332]

Will u recommend any book ?

[@rajasdataengineering7585]

Spark definitive guide is good to understand spark internals

[@abhishekjoshi48]

Hello Sir, Thank you for video, I have requirements where I need to change datatype of column instead of long it should be string what would be the possible ways?

[@rajasdataengineering7585]

Hi Abhishek, you can use cast method in pyspark