Advancing Spark - User Access & Power BI with Databricks

Hot on the heels of Microsoft Ignite 2020 we have the preview release of the Power BI connector specifically for Databricks. This brings the ability to connect directly using Azure Active Directory, which adds a far tighter security model & better user experience.
This week, Simon looks at applying table access control, granting limited select to a user and connecting via the new Power BI connector!
For details on the Databricks connector, see - docs.microsoft.com/en-us/azur...
As usual, don't forget to like & subscribe, and stop by www.advancinganalytics.co.uk/... for more details about how we can help you make the most of Azure Databricks!

[@TechWorkbench]

Great video, as always! Would love to see a deep dive into lake level security and access patterns from Databricks

[@AdvancingAnalytics]

Working on it - need to do some baselining of whether the security abstractions still cause a slow-down of querying times. When I've put it through it's paces, I'll pop a Databricks security patterns vid out :) Si

[@sergiocoutinho6133]

Hi Simon .. a great video. Thank you very much for the informations. I have noticed that i can only enable "table access control" feature when I configure "cluster mode" on "High Concurrence"

[@sarveshkumar-rt1nn]

like 100, awesome video

[@dhanunjay3510]

as usually awesome explanation, is there any way to connect or view/access my databricks cluster logs through powerbi.

[@shubheshswain5480]

Hi, I liked your explanation in the video. I have a question regarding Power BI with Active Directory. When I import the data from the Active Directory into Power BI, I see many missing values but when I check the same data in the Active Directory, the data is available there. Am I doing something wrong while importing the data from Active Directory?

[@amjds1341]

Can attach to cluster means ?

[@sid0000009]

If we have Table Level access and Pass through authentication enabled on a cluster. Which one would be over-ridden? thanks

[@AdvancingAnalytics]

They're incompatible - you can't have both settings on a single cluster. Should probably have made that clear in the video!

[@akhilannan]

If we create a Databricks DirectQuery PowerBI report, and publish it to Service and share it with a user, will the users credentials get automatically passed through and they will be able to see data from tables they have access to??

[@AdvancingAnalytics]

Gooood question! Haven't tried honestly. With other AD Passthrough connections, especially with DirectQuery, it'll pick up the relevant consumer's AD token. Whether the brand new Databricks connector has implemented the same pattern, I couldn't say. Worth testing out though! Simon

[@imeneboukari5318]

Hello ! I have the same question, did you find an answer please ?

[@guilhermelira88]

It's possible define an access control in a table to a especific user?

[@guilhermelira88]

If you have some video that you teach this please tell me.

[@umarhussain9334]

56 GB cluster! Holy moly what's the cost for that per month?

[@AdvancingAnalytics]

Ooh! I hadn't noticed that - looks like the default VM size switches when you select High Concurrency, that's cheeky! I only had that specific cluster turned on for an hour or so - if you left it running that's about £2k a month full time. Just business hours would be around £800 (200 hours p/month). So not insane, but more than you would pay if you paid attention to VM selection!! Simon

[@Manisood001]

can i access hive delta tables in power bi without using interactive cluster

[@AdvancingAnalytics]

Nope! Interactive cluster (or SQL Analytics endpoint) is currently required to view/interact with the hive tables!

[@Manisood001]

@@AdvancingAnalytics I want to build a solution so that i dont constantly run interactive cluster for my bi users.I want cluster tostart and stop after powerbi gemerates reports

[@AdvancingAnalytics]

@@Manisood001 If you're using an import model over an interactive cluster, the cluster will start automatically when power bi tries to refresh, then you can automatically terminate after 10 mins of inactivity. So cached Power Bi models don't need the interactive cluster, just direct query. If you need to use direct query, your users will either have to wait for the cluster, or have a cluster always running (during business hours)

[@Manisood001]

Thanks, highly appreciate, just wondering why your channel is underrated. please add azure data factory and azure data ops, azure powershell scripts to automate ARM TEMPLATES, METADATA DRIVEN INGESTION, CONFIG TABLES, AZURE DATABRICKS TO SYNAPSE , GIT CI CD CONTROLD IN DATABRICKS AND BIGTER PICTURE OF A DATA PLATFORM INCLUDING DOMAIN Thanks for your endless effort