What is a dataflow? | Practical example included

Join this channel membership to get access to all the recorded bites as they become available:
/ @curbalen
Here you can download all the pbix files: curbal.com/don...
SUBSCRIBE to learn more about Power and Excel BI!
/ @curbalen
Our PLAYLISTS:
Join our DAX Fridays! Series: goo.gl/FtUWUX
Power BI dashboards for beginners: goo.gl/9YzyDP
Power BI Tips & Tricks: goo.gl/H6kUbP
Power Bi and Google Analytics: goo.gl/ZNsY8l
☼☼☼☼☼☼☼☼☼☼
POWER BI COURSES:
Want to learn Power BI? How about you take one of our courses? Here you can find the available courses:
curbal.com/cou...
☼☼☼☼☼☼☼☼☼☼
ABOUT CURBAL:
Website: www.curbal.com
Contact us: www.curbal.com/...
QUESTIONS? COMMENTS? SUGGESTIONS? You’ll find me here:
Linkedin ► goo.gl/3VW6Ky
Mastodon ► @[e-mail]
Twitter ► @curbalen
Facebook ► goo.gl/bME2sB
#CURBAL #SUBSCRIBE

[@HachiAdachi]

We use Dataflows primarily to store 'flat files' data that require on-premises data gateway for refreshing. We have some csv files on our on-premises network drives that are used and transformed in so many different ways by many semantic models/reports. Instead of having the data go through the gateway each time data models are refreshed, each csv file is transmitted into shared a dataflow only once a day independently, and the data models can connect to the dataflow without us having to worry about the capacity of the gateway server or the Internet bandwidth.

[@kaml005]

how is this different from a powerbi report which does the same. Add source,do transformation, refresh daily

[@m.rosariomartinezmiguelez9015]

Buenas tardes :Un gusto volver a verte :Te escucho con mucha atención 🎉❤

[@CurbalEN]

Esta vez no me viste!

[@m.rosariomartinezmiguelez9015]

No ,pero con oírte ya me consuelo

[@CurbalEN]

@m.rosariomartinezmiguelez9015 Os llamo esta semana ❤️

[@DataHacker]

Unless I am missing something, I would always go with gen2 + store in lakehouse, if capability is available. Different storage method, sql query or even use lakehouse semantic model if that is someone's thing 😊

[@CurbalEN]

You need a fabric license, not everyone has one, that is why I demoed it with gen 1 :)

[@DataHacker]

@CurbalEN Ok, fair enough :) Although I would suspect majority of serious users will have premium capacity of some sorts.

[@HachiAdachi]

@@DataHacker 😳 I must have been doing this wrong because I've been managing an analytics team for a mid-size company only with ≈180 Pro licenses for >5 years... or I must not be serious enough? While I would LOVE to have the Fabric capacity at our disposal, most things are being easily handled without it, so it is difficult to justify the added costs to the executives. I would suspect that many other 'serious' Power BI developers are on the same boat.

[@DataHacker]

@HachiAdachi I am sorry, I did not mean to insult anyone :) I guess every org will be different. I am probably having too much of big corp bias. We have loads of users, so at the time of adoption it made real sense, as viewers could view reports with free license. Now we have our data stored in Fabric, so that is another good use for us. If you are able to fit under Pro license and you do not need the other features (however cool they may be), then yeah, why pay more. And Fabric is not perfect, to be fair...

[@HachiAdachi]

No worries, @@DataHacker . I often feel like I have to hack around to compensate for the lack of the premium tools, so in reality, I DO need these features. I just need to be able persuade these executives somehow...

[@latishshelukar8906]

Hi Ruth, I have one use case for my project. I need semantic model refresh history in a report. After some analysis, I found rest api 'Get Refresh History in Group'. But how can we create report using this? In short, how to integrate it with Power BI. There is some Power BI Automate solution for this. But I am not able to get it. Can you create one video on this if possible, that will be helpful.

[@davida.taylor8444]

What's the benefit of using this instead of including this in your M Code and scheduling refreshes once your file is published?

[@CurbalEN]

A dataflow is used purely to move and transform data. It does not have modelling or dataviz capabilities and you can move data to other storages like data warehouses, lakehouses etc…

[@DataHacker]

@@davida.taylor8444 I would say, if you need an output that is going to be reused multiple times, a single source of truth, then you would potentially go with dataflow. If you have a very complex model, sometimes you may want to offload some of m code to boost refresh performance. There are other uses too. But for the most part, you should be ok with just a standalone PBI model

[@imconfused1237]

Dataflows are perfect for efficient reuse. Almost all of us use a custom Date Table. With a Dataflow, you create it once and then connect to it. And what’s more, if you ever need to modify that Date Table, you do it once and it gets propagated to all connected models. As a practical example each month, one of my models connects to 25 Dataflows that handle almost 10m records. With Dataflows I’m pushing that level of ETL upstream and in doing so avoid choking the performance of my pbix. Sadly our workspace doesn’t allow Dataflows with external queries. I think it’s a licence thing. Therefore I have to cheat and nest my functions within the M code instead (…and usually have a little dance with the Formula Firewall error 🤭)

[@davida.taylor8444]

@@imconfused1237 Thanks, that's helpful.