Power BI - Accessing Azure Data Lake Gen 2 Files (11) - Hands On Demo!

Power BI and Azure Data Lake Storage is the key to the future of GREAT BI! Here I show you how to connect your Dashboards and Reports to Azure Storage Gen 2 the cutting edge type of Azure data storage and where current and future data lakes will reside. Also, through a very simple example, I teach you about the basic organization of files in a data lake and how powerful this non-ETL, non-database approach to data storage makes it so powerful. Download Sample Files: sqlcoop-my.sha...
00:00 Create Gen2 Data Lake Storage Account and Container
01:10 Upload Files to Container
03:00 Review files' contents
04:08 Power BI - Get Data from Gen2 File Container and Sales folder Url
06:45 Get Storage Account Key
07:02 Error: Access to the resource is forbidden and Role Based Access Control
11:02 Connection Succeeded
11:53 Combine data from multiple files
13:00 Create sample report from data
14:30 Add additional new subfolders and data to container
17:00 Refresh report with new data
( / greg-c-coopman-b07b54103 )

[@tinashechinyati6823]

Thanks Coop. You saved me a great deal from sinking all my data in one folder. I thought i couldn't read from a folder structure. Thanks a lot

[@CoopmanGreg]

No problem. Glad it helped! ...thanks for the comment

[@ranafarhanmajeed3352]

I specially come to youtube from web to give you a like and comment ... so nicely explain each and every bit thanks

[@CoopmanGreg]

Cool, sure glad you liked it!

[@ralber01000]

Thank you!! Thank you!! Thank you!! for purposely going through permission issues.. Great Presentation

[@CoopmanGreg]

You are so welcome!

[@johngriffiths4636]

Nice. Thanks Coop. It pooped up in recommended and I thought woah ! Azure, this is going to be all techy. Nope! Simples. Point and click configuration and menus. I work in Power BI and the company I work for is just setting up Azure Gen 2 and a data warehouse, and I know some basics, but this was great to see behind the curtain without having to do a 100 hour course to find the bit that showed me how a Power BI developer would access the data in an Azure data lake. Many thanks.

[@CoopmanGreg]

Great John, I'm glad it helped. I also work on data lakes and gen2 files and knew I had to be able to access those files from Power BI.

[@germanshepherd-simba6177]

Literally I was struggling to understand this all in power query, finally got the idea how these parameters created Thanks a ton to you!!!!!!

[@CoopmanGreg]

You're very welcome!

[@peterdolina850]

This is a very well done video, that approach where possibe blocks are clearly shown makes it in my view on of the best if not the best on the topic.

[@CoopmanGreg]

Thanks alot, glad it helped!

[@lukeroberts70]

Fantastic, you have saved me a lot of time! I was really stuck on this, great video, thanks!!

[@CoopmanGreg]

Glad it helped!

[@adityagaikwad943]

You are awesome!!! Thanks a ton man!

[@CoopmanGreg]

Happy to help!

[@Awachit1]

Hi, great video What if we have to select only one folder?

[@HenryTsang]

Thanks for pointing out where to get that URL. I assumed it was the URL under the Container Properties but that was not it.

[@CoopmanGreg]

You, are most welcome.

[@rogerren]

@@CoopmanGreg great video, thank you. Do you have a video showing how to schedule power bi data refresh sourcing from the azure data lake?

[@CoopmanGreg]

@@rogerren Sorry Roger, I have not made one for that. Thanks for watching, though.

[@kalyanbabu8596]

Hello Greg, I have one question, can we make this connection string for the container as dynamic? I'm working on a requirement where the storage account has multiple containers and these container names has to be passed from ADF pipeline to Power BI for to read multiple files from the container subfolders. Is this possible collecting container names dynamically from ADF to Power BI?

[@venkyreddy4138]

is it possible to store power bi dataset to azure data factory if yes make a video for this

[@vineetpatil86]

Thanks for sharing Coopman! Also if you can share,when we configure data source on PowerBI services for automatic refresh of the source files, do we have to create multiple connection for different files? any idea/video on configuring data gateway for ADLS2 on powerbi services for refreshes!

[@CoopmanGreg]

You are welcome. To answer your question, I would look into the Incremental Refresh in Power BI Premium. Unfortunately, I believe Pro and Desktop may not have what you are looking for.

[@vineetpatil86]

Thanks coop for reply, may be I did not explain it correctly, when we take powerbi report to powerbi services on cloud, then When we need to refresh the report that time we configure data gateway connection, so my question is , do we have to configure data gateway for each file source or can we configure it for folder where azure files resides and have single connection used for Multiple files within same folder Thanks

[@olakunleoyeleye2224]

Thank you Coop. Please can you help me on how to connect power bi to gen2. I connected to gen1 by “Adl:// url “ meta [ isparameterquery =true, type=any, isparameterqueryrequired=true] How do I connect to gen2 from M query.

[@CoopmanGreg]

Sorry, I have never customized the M query.

[@olakunleoyeleye2224]

@@CoopmanGreg thank you

[@ankitsharma1943]

Do we need to install gateway for excel file stored in adls gen 2 for refreshing the data

[@CoopmanGreg]

You should not.

[@umapathi04]

Hi, I am using Azure Data Lake Storage Gen 2 as my source for Power BI and i am selecting CDM Folder View in Data view and create reports using and when i refresh in Power BI desktop it is refreshing successfully, But when i publish this report in Power BI service and try to refresh in Power Bi service i am getting the error while refreshing. Could you please help on this. Thanks in Advance!

[@CoopmanGreg]

Look into gateways. Maybe you need one.

[@ashishpandey9429]

Hello sir, can we connect powerbi pro to adls gen2 ?

[@CoopmanGreg]

Yes, see kzbin.info/www/bejne/pqjWgK1maqabldU

[@ashishpandey9429]

@@CoopmanGreg Thank you for responding! Also can we read parquet file in PowerBI from Gen2

[@CoopmanGreg]

In PowerBI there is a Parquest connector under File in the Get Data option. It gives you a URL to point to. That is as far as I can point you on that matter, though. Thanks