Parameters and Variables in Azure Data Factory

Learn more: • Azure Data Factory Tut...
🔗 Azure Data Factory Tutorial 5 | How to Use Parameters and Variables 🔗
Ready to elevate your Azure Data Factory skills? 🚀 Join us in this tutorial as we unravel the powerful concepts of Parameters and Variables in Azure Data Factory. Whether you're new to data orchestration or a seasoned pro, this video has valuable insights for everyone.
🎯 Here's what you'll learn:
🔹 Conceptual Clarity: We'll start by demystifying Parameters and Variables, providing you with a solid conceptual foundation.
🔹 Parameters in Action: Witness how Parameters empower you to customize and flexibly control your data pipelines. We'll guide you through real-world scenarios where Parameters shine.
🔹 Variables Unleashed: Dive into the dynamic world of Variables. Learn how they enable data manipulation, dynamic storage, and flow control within your pipelines.
🔹 Hands-on Demonstrations: Watch practical walkthroughs in the Azure portal, where we'll implement Parameters and Variables, showcasing their real-world applications.
💡 By the end of this tutorial, you'll be well-equipped to harness the full potential of Parameters and Variables, making your data workflows more dynamic and adaptable.
📢 Don't miss out on this opportunity to enhance your Azure Data Factory expertise! If you find this tutorial insightful, give it a thumbs up 👍, subscribe to our channel, and turn on notifications for the latest updates. Feel free to share your questions or suggestions in the comments below.
🔗 Link to the material used in the demos:
drive.google.c...
#AzureDataFactory #DataIntegration #Tutorial #ParametersAndVariables #DataOrchestration #AzurePortal #CloudComputing

[@shobhitsharma2137]

Please continue making these

[@AleksiPartanenTech]

Thanks, I will! :)

[@jarker3107]

You deserve much more views! Please keep on creating practical and straight to the point videos like this.

[@AleksiPartanenTech]

Thanks! I will keep making more videos :)

[@janisdata7874]

@jarker3107 I agree. @AleksiPartanenTech Erinomaisia opastuksia, kiitos!

[@AleksiPartanenTech]

@@janisdata7874 kiitos!

[@timothywald9079]

I love your content but learn better interactive....is there any training you would recommend that's instructor led? I'm a software manager and i want to learn adf for my job. Been trying to follow but learn best from doing. Any suggestions?

[@AleksiPartanenTech]

Thanks for your feedback and question! I don't have any particular course in mind but I think udemy has many courses on ADF to choose from. I don't know any courses that would have a live instructor that would be ready to help with exercises and questions. Almost all of my ADF videos are tutorials at the same time that you can do along with the video and after that I would recommend to modify the pipeline a bit to try out what happens if x in the pipeline is changed. In my opinion the best way to learn ADF is to do pipelines that are going to actual use and have a mentor who would help and do some quality control.

[@davidvinh1786]

I'm a newbie learn a lot from your videos.Thank you so much for the videos. But How can you loop through thousands of files?

[@AleksiPartanenTech]

Glad to hear that you have learned a lot from my videos! :) For many files I would recommend using wildcards (e.g., *.csv) when copying them because copying thousands of files one by one can become very costly very fast.

[@orangehouse1105]

Hi aleksi, may i ask some question, if i want to use variable and call the variable in data copy task, its possible ? For example in ssis i have variable ‘var,, just declare the variable, and then in data flow task i just using query like this ‘select * from mytable where descriptions =‘var’

[@AleksiPartanenTech]

This depends on your source. If you have SQL database type source you can use query when reading data. Then you would use variables in dynamic expression like this: @concat('select * from dbo.table1 where col1=',variables('var1')). Hope this helps you :)

[@orangehouse1105]

@@AleksiPartanenTech thanks somuch about your answer very helping me 😁

[@Chikookathalu286]

Hi Aleski nice explanation. Please can we create one practical session on parameters separately showing difference how it is diferent @piplinelevel,@linkedservicelevel@datasets level so it would be more helpful..😇😇😇and which level situations w e need to use which one?

[@AleksiPartanenTech]

Thanks for you comment and feedback! I will consider if I do a follow up video where I dive deeper into parameters. Sorry for a late reply since I have been on a holiday for few weeks.

[@bananaboydan3642]

Great video, but Im trying to copy data from parquet files into an Azure SQL DB from blob, which has a storage event trigger. How will the schemas map if Im using a generic parameter and not an actual file or table schema?

[@AleksiPartanenTech]

Thanks for your question! Hmm I don't know if I understood the situation correctly but you don't have to specify the schema when copying data to SQL DB. In that case it uses the columns in the file for the table structure. Also, event trigger itself has some properties that you can use in your logic and pass them as parameters for the pipeline if that is something that you are after here.

[@bananaboydan3642]

@@AleksiPartanenTech Do you have a linkedin or some other social media, the error I continue having is a lot to explain in the comment section

[@bananaboydan3642]

@@AleksiPartanenTech The copy activity wont ccopy my actual data to the SQL table, it copies the schema alone but my table is empty. I check the file itself and it has data.