In this video will discuss about , how we are going to perform data validation with pyspark Dynamically Data Sources Link: drive.google.com/drive/folder... #pyspark #databricks #dataanalytics #data #dataengineering
Пікірлер: 13
@mohitupadhayay1439Ай бұрын
Amazing content. Keep a playlist for Real time scenarios for Industry.
@ajaykiranchundi99792 ай бұрын
Very helpful! Thank you
@listentoyourheart456 ай бұрын
Nice
@vamshimerugu61842 ай бұрын
Great explanation ❤.Keep upload more content on pyspark
@DataSpark452 ай бұрын
Thank you, I will
@ArabindaMohapatraАй бұрын
I just started watching this playlist. I'm hoping to learn how to deal with schema-related issues in real time.Thanks
@DataSpark45Ай бұрын
Thanks a million bro
@skateforlife36796 ай бұрын
Cool, but is it like this every time ? Like you have a reference df containing all columns and file name / path and you have to iterate over it to see if its matching ?
@lokeswarreddyvalluru59186 ай бұрын
Yes
@World_Exploror4 ай бұрын
how did you define reference_df and control_df
@DataSpark454 ай бұрын
we defined as a table in any DataBase. As of know i used them as a csv
@OmkarGurme4 ай бұрын
while working with databricks we dont need to start a spark session right ?
@DataSpark453 ай бұрын
No need brother, we can continue with out defining spark session, i just kept for practice