Microsoft Fabric: How to load data in Lakehouse using Data Pipeline | End to End Scenario

[@123jamalq]

Amit aap ka bohat shukriya, doing a great job.

[@AmitChandak]

Thanks a ton 😀

[@mohammadaamirkhan7830]

Thank you once again for this nice video. I learned many things new today.

[@AmitChandak]

So nice of you. Thanks 🙏

[@shrikantthakur7127]

Thanks For Sharing Fabric Knowledge

[@AmitChandak]

It's my pleasure 🙏

[@rupalisingh4340]

Hello Amit, how can we copy data from a sql server to lakehouse using a pipeline in fabric As I'm trying to do that but getting an error as dynamic link library routine failed.

[@AmitChandak]

It should be possible with March 2024 update with latest version of on-premise gateway. Now Integrate On-Premise Data with Microsoft Fabric Using Data Pipelines: On-Premise Gateway March 2024 Update: kzbin.info/www/bejne/qYfIgamQhM-npdE

[@shafa7668]

Hey Amit, Super usefu video mate. Much appreciated for doing it. How do you schedule the pipeline based on a trigger?

[@AmitChandak]

Thanks, As of now you can schedule it, using the schedule icon. For Trigger can explain little bit more, Example - SSIS triggers the pipeline

[@flyincarpet1338]

one suggestion, can you use the numbering ? 01, 02, 03 etc in front of each video its easier to follow Thanks for the videos

[@AmitChandak]

Thanks, I will try to add numbers. Please find the series in which all videos are in order Mastering Microsoft Fabric 50+ Videos: kzbin.info/www/bejne/pl7ZYXxriJKsmNU

[@flyincarpet1338]

@@AmitChandak Thanks for your quick reply Can you also make CICD integration with azure devops or github into Fabric?

[@manfredmbedzi8137]

Hi, Thankyou so much for this. Doyou have an updated video with the options : - Load Settings - Column Mapping - And Enable Partitions

[@AmitChandak]

Hi Payal, Thanks for letting me know about the pop-up videos at the end. I understand how distracting they can be. I've already shortened them to just 5 seconds, but I'm happy to completely remove them if they're still causing issues. By the way, this particular video is part of a larger series. Here's the playlist link in case you want to explore other related topics quickly: kzbin.info/www/bejne/jZfVYoqearirra8

[@faheemiftikhar706]

Hi! Please can you make a video on loading the data from other sources such as postgresql by using dataflow or pipelines?

[@AmitChandak]

I have used PostgreSQL in incremental load video Microsoft Fabric Part 15: Incremental Data Load(ETL) for Warehouse using Dataflow Gen 2, SQL Procedure, Data Pipeline: kzbin.info/www/bejne/o6Gpg6GQd72mrLM

[@faheemiftikhar706]

​@@AmitChandak Thanks for fast response and sharing your knowledge.

[@Wolf_claws]

hi sir i was watching your video just wanna ask why did you used http extension for copying the data from the github

[@AmitChandak]

GitHub website is https. I am just making sure I am using raw link not the blob link.

[@ben79341]

I found copy task really expensive CU wise. billing out a minute even if it takes a second or 2.,

[@AmitChandak]

Can you share some more details or ping me on LinkedIn and share some screenshot- www.linkedin.com/in/amitchandak78/