Data Lake in a Day - Lab 3 Processing the data model with data flows

[@pranavkumar9030]

Hi Dave you put a great effort in explaining the details, Thanks for that. I follow along and when I finish and run the data flow pipeline I got the aggregated folders as we designed in the data flow, apart from this I also got some empty files with the same name as in the folders created by data flow, the same thing happened in your case as well, may I know why it has been created.

[@pranavkumar9030]

you can check at timestamp 24.40

[@DaveDoesDemos]

Hi Pranav thanks for the comment and glad you're enjoying the series. These files are just metadata used by ADF and can be ignored. Once the process is complete you can remove them if you really want to, but there's no benefit to doing so as they are very small. When using Parquet format I believe they hold more information, but with CSV it's all self contained.

[@pranavkumar9030]

@@DaveDoesDemos Thanks for your Reply