I’d like to see a “data solution in 10 minutes” built in Synapse. Would be really powerful if that included SQL and Python to demonstrate the flexibility.
@daddogsdata2 жыл бұрын
Great idea! Please!!
@guimatheus922 жыл бұрын
Yes, please!
@hendrik-stack222 жыл бұрын
On the one hand this format could cover topics like Azure Synapse in conjunction with Power BI and on the other hand Synapse with SQL/Python/PySpark etc. I would be really interested in this. Also, do you recommend other youtube channels for Azure Synapse? I think it could be hard to keep up-to-date with the latest updates in Power BI AND Azure Synapse.
@heshamh962 жыл бұрын
If u ever found it ... Please tag me
@akthar32 жыл бұрын
I dont think azure synapse is a magic wand (correct me). All I understand is that it is an ETL and orchestration tool and more . You still need to do that development work!
@jitendrabisht62952 жыл бұрын
I was waiting for this video.👍 Great work
@GuyInACube2 жыл бұрын
Woot! Appreciate that! 👊
@KahanDataSolutions2 жыл бұрын
Really helpful overview. For some reason I always assumed Synapse was just Azure's cloud database offering - didn't realize it was all of these other concepts too. Thank you!
@karthi-h2y2 жыл бұрын
Glad that you are going to do a detailed series on Azure Synapse. I just joined Microsoft and i have a opportunity to work on Azure Synapse and your detailed explanation will surely help me
@nclsduran2 жыл бұрын
I would LOVE you explain spark! The way you explain the topics are great!!
@prawnyowl89092 жыл бұрын
Is there a good YT channel for learning Synapse, like Guy in a Cube? Where do I start?
@GuyInACube2 жыл бұрын
The plan is Guy in a Cube :) just getting started and will continue this journey. 👊
@pawelpotasinski90922 жыл бұрын
Besides Guy in a Cube (love you guys!) you can check these resources to learn Azure Synapse: - Azure Synapse Analytics KZbin channel (see the Essentials playlist there): aka.ms/SynapseKZbin - The Get Started in 60 Minutes video: aka.ms/SynapseGetStarted - Azure Synapse Analytics Toolkit (a set of documents for quick start with different aspects of Azure Synapse): aka.ms/SynapseToolkit - KZbin channel of Data Toboggan community events (focused on Azure Synapse): kzbin.info - Synapse Practitioner site (lots of free online learning paths): aka.ms/SynapsePractitioner - Learning paths on Microsoft Learn site: aka.ms/SynapseLearningPaths And, of course, stay tuned for more awesome Azure Synapse videos from Guy in a Cube ;-)
@antoniocolombo72042 жыл бұрын
In brief, as per my understanding: - Power BI provides strong ETL with Power Query and even data modeling. Moreover Power BI golden rule is to use a star schema. - ASA for sure provides the all the tools for datawarehose while lacks visulization features as far as I know. For DW, PBI star schema (e.g. 1:n dim to fact relashionship) is not striclty a must Moreover, because of the tons connector and modeling capabilities available in PBI, it can be used to pull and shape data from existing data marts and provide "enterprise" solutions just within PBI So what is/are the trigger(s) point(s) that suggest to move from pure PBI solution to ASA+PBI? I know, depends... ;-))))))))))
@hazmatsuiter2 жыл бұрын
Doing ETL in Power BI is great if you only need to load to Power BI, if you need to get that data anywhere else it is pretty poor (Dataflows can be created to load to Dataverse tables but that also has its own limitations; Dataverse also has an Azure Synapse Link feature)
@antongerasimov22982 жыл бұрын
Etl in power bi is very simple. If you want to be more powerful you should look at synapse
@egorshiyanov12062 жыл бұрын
Guys, you are amazing! Or Insane Amazing I should say! Great intro and presentation of ASA! Keep up the good work 👍 Love your channel!
@joshash59442 жыл бұрын
The dedicated sql pool in synapse can get very expensive. If your data volumes aren't in the terabyte range then you probably don't need a distributed architecture like synapse. I doubt the tooling or development experience in synapse would be a reason for any organisation to use it... azure sql + data factory gives you the same things The distributed nature of synapse is the whole reason it exists. This isn't a technology designed for 10GB databases :)
@jbonomo2 жыл бұрын
Glad to see someone say this. So much hype around Synapse but it is not cost effective for small businesses without TBs of data
@pawelpotasinski90922 жыл бұрын
@@jbonomo have you tried serverless SQL pool in Azure Synapse? I think the combination of this engine and Power BI can meet the requirements for small business analytical use cases when data is not that large and should not drain the budget. Synapse is not a single engine (right now there are 4 of them - besides dedicated SQL pools also: serverless SQL pools, Spark and Data Explorer - all addressing different use cases and having different pricing models).
@egbertdejong63772 жыл бұрын
Looking forward to this series. Hope using Databricks vs Synapse is also covered in a way.
@samanthapikus2134 Жыл бұрын
Hellooooo... Love your videos... Always so filled with the right kind of Energy. You mentioned data orchestration and learning the clicks... What does that mean?...How do I start going from Power BI analyst to using Synapse and become more of a data engineer?
@ricmmf Жыл бұрын
I come from the tradicional DW / BI and i'd love a session that would show the "state of the art", this is, ok, i have a DW on synapse, i need to pick these log times from machines to do some analysis, and some Excel files to do another job, and so kind of seeing, by example, all (or most) parts of synapse working and connecting together...
@dan7582 Жыл бұрын
Does you guys have some sort of commercial relationship with Microsoft and Azure? If yes you probably should make a disclaimer at the beginning of the video.
@ncbshiva2 жыл бұрын
I want to know the difference between dedicate pool and serverless in Azure Synapse Analytics.
@GuyInACube2 жыл бұрын
Stay tuned 👊
@SamuelMohammedOmer2 жыл бұрын
We are moving to ASA recently and couldn't be happier. I hope you guys will have more vids on it in the future.
@GuyInACube2 жыл бұрын
Great to hear! The plan is to drop videos on Tuesdays going forward.
@marlucaplus2 жыл бұрын
Amazing video guys! It would be nice to know the difference between the power bi in synapse and the regular one. what can I do in synapse and what I can't. and of course why would I use the one in synapse...
@antongerasimov22982 жыл бұрын
let's start to do videos about Synapse!
@TrevorMatthews2 жыл бұрын
It'd be nice if there was a more affordable tier to explore Synapse. I was just looking on the Azure pricing calculator and looks like the cheapest monthly fee is a couple thousand dollars which makes it pretty tough to get much hands on exposure.
@g.michaelaguirre89132 жыл бұрын
1) Migration. e.g. migrating on prem SQL data and SSIS packages (in bulk) 2) Homogenizing SQL & Power BI w/ ML libraries
@pawelpotasinski90922 жыл бұрын
Love it! Really happy to see you guys recording this series. Looks like it's going to be one of these "must see on Azure Synapse" materials we will share with the community!
@Phoenixspin2 жыл бұрын
I listened to this but I have no idea of whether or how I could use this as a normal mortal user.
@sbksai71422 жыл бұрын
1) Major drawbacks of Spark Pool in Synapse vs Data Bricks and vice versa? Which is better? When we are doing POC initially to evaluate Spark pool in Synapse, we have observed lot of drawbacks when compared to Data bricks. I would love hear from you with all the recent updates, can Spark pool perform much beyond than Data bricks? If yes on what basis? 2) Why not Azure Purview in Synpase? We can do governance directly there itself instead of a separate service? Any reason for making it separate? 3) One major drawback I observed is, sql pool can't support three part naming and we need to create separate schemas in the same sql pool to differentiate instead of creating different sql pool which is a costly operation. But it all together doesn't correlate with the traditional EDW modeling we do like we keep separate staging, logging, edw databases etc. 4) Why can't we delete duplicates direcly querying CTE pointing to Original table if the sharding pattern is other than Hash? We have alternatives to perform this task. But want to know the reason for why can't we directly delete using CTE? 5) Any concept of queue while submitting Spark jobs in Spark Pool? How much time will it take to spin up the cluster and run the jobs that are in queue? Comparison with data bricks make us learn better. Please help us in letting know the above queries. Will post further questions once they arise.
@Mim_BI2 жыл бұрын
I know why you are doing this, but I hope you keep the message very clear, PowerBI is still an independent BI tool and works with any platform not only synapse
@Wtreadway01292 жыл бұрын
I’d love to know about partitions on the large data tables.
@mauriziofaini92522 жыл бұрын
I wanna know more! YES!!!!!
@Griffindor21 Жыл бұрын
So does it actually do what it says, analytics? Is it the same as SAS Viya where you can do exploratory and predictive analytics?
@Real282 жыл бұрын
I JUST got access to our Azure instance Friday and was starting to look for resources to watch. Then this video comes up. You guys are the best and your timing is _impeccable_ .
@chrisretsin70682 жыл бұрын
Great videos, as always. I have a couple of question regarding lake house. 1/ Our IT department is moving the analytical data to the Azure cloud. Upon this move they want to add a non-azure product for data virtualization. Does this make sense? What are the azure possibilities here? 2/ what about erroneous data in the data lake, how the identify the erroneous data, correct it and restart the process without the erroneous data 3/ what about showing data in reports containing enough data in order to avoid possibility to identify the source
@rachwalskimarcin2 жыл бұрын
Why costs are so high?
@vkp13102 жыл бұрын
Hi Guys, I manage a IT Data systems and currently we use the traditional methods of ETL/ELT(ADF) and then use Azure SQL to process and store the data. How easy it is for me to move from this to AZURE synapse? Is it worth the transformation or this needs to be setup from the scratch. Thank you. VKP
@ericmclaren4632 жыл бұрын
I do not recommend Azure synapse. When MS renames a product there is a reason why!
@vishnumuthyaboina75462 жыл бұрын
I don't have Azure cloud experience can I directly jump in and learn ASA from scratch(including basics of Azure)
@GuyInACube2 жыл бұрын
It will require a few clicks and an Azure Subscription. Our goal is to help you in that journey with getting started and moving beyond.
@alfredsfutterkiste75342 жыл бұрын
So happy you guys are branching out to Synapse! Very much looking forward to more videos with the GiaC production quality on this topic!
@ahmadsawal29562 жыл бұрын
Yes Please, we need detailed videos about Synapse with databriks, power bi etc. all the Azure data services please
@GuyInACube2 жыл бұрын
Just starting the journey. Looking to get the data journey to make sense and looking at the solution instead of the individual silos.
@kopalsrivastava222 Жыл бұрын
more use cases on Azure Synapse Real-time Analytics
@AlvinRyellPrada2 жыл бұрын
Is it possible to cover Azure Machine Learning please?
@rudroroy10542 жыл бұрын
So Synapse integrates ADL, ADF, SQL WH/Spark & Power BI service. Can we use other data storage services like MySQL Managed instances and PostgreSQL?
@the_invisible__2 жыл бұрын
Yup....You can use one of the Cosmos DB API's of MySQL, Postgres
@YesAG2 жыл бұрын
would you outline a use case to migrate from DB2 to Synapse vs MI
@brianpclab2 жыл бұрын
I did a POC attempting to connect a Power BI Dataflow to a partitioned ADLS Gen 2. Dataflow connector would not work, Then created a database and CETAS table in serverless SQL. Using Azure SQL connector works but the issue was that if you have partitions like puiYear=2022/puMonth=05/puDay=03 when you append the Gen 2 storage with another days worth of data it will never be seen in Power BI nor in SSMS. Finally if you use views instead of CETAS tables, when you append additional days to your partition all you have to do is refresh the DataFrame and DataSet and everything is properly refreshed. The appended data also shows up right away if you use SSMS to query the view. Problem is that this is NOT documented anywhere that I can find. Would really appreciate it if you could give us best practices for using partitioned Gen 2 storage with Power BI. I am looking for confirmation that this is the recommended method and that we will not have issues down the road,
@riddlex2 жыл бұрын
I'm a Data Analyst with knowledge in SQL, Python and Excel. I want to get started with Azure Analytics. So, Azure Synapse would be enough for me or do I need something alongside that?
@prasadparab5638 Жыл бұрын
Thanks for the valuable insights... can we have high level sample end to end solution done using Azure synapse with its integration with Power BI Thanks
@CalvinPuah2 жыл бұрын
now that you've mentioned it.... Power BI with Azure Synapse, managed through DevOps and CI/CD would be a good start :P If possible
@davidwilliams85272 жыл бұрын
When working with Azure Synapse Analytics, is it still best practice to work in different environments for DEV -> TEST -> PROD? Do DBA's still have a role in helping to promote code between those different environments? Also, if I use Azure Synapse, is there a need to use Azure Data Factory or Azure Data Lake independently, or is that all bundles into Synapse and does Synapse "replace" the independent use of those other products?
@powerbiprodesigner79492 жыл бұрын
Hi. On the SAAS platform like the Azure SQL MI / synapse, the debug mode is not available. this is making it very difficult to debug stored procedures which sometimes can get very complicated. We have been told by the MS Support team that the components required for debug are not available in the SAAS platform. I had to set up a VM, install SQL Developer Edition and do my work. Once the SP was ok, I deployed it to Azure SQL MI. Does the ASA overcome this limitation ?
@houstonho45522 жыл бұрын
I am having 2 projects on Synapse. The first one is extracting data from Dynamics AX 2012 on-premises SQL into synapse, trasnform the data into star-schema in Synapse, which being used on Power BI (BI report maintained by IT developers). The second one is extracting data from both Dynamics 365 Customer Engagement and Finance Operations thru the Azure Synapse Link (Data Lake connector), trasnform the data into star-schema in Synapse, then populate into Azure Analysis Service and into Power BI (maintained by >100 business users and self service thru ASA). I believe each component has a bit overlapping featurs however they have different target audience, different technology at the back, and different level of user friendiness. In fact, i can't find an official doc to differentiate them. It would be great if you could share you point of view. Thanks.
@cooky123 Жыл бұрын
I'd like to know more about when to use Synapse Analytics vs DataBricks
@josuelabaut99692 жыл бұрын
CI/CD with synapse studio. Can we use synapse studio with one existing Data Factory project (Migration)
@adanielmacedo2 жыл бұрын
Hi guys thanks for your videos, you are amazing... About Synapse, where can I get a free way for testing, with the a Azure account, you pay for Data Factory for example. Best wishes 😃
@priyankaadhainge83602 жыл бұрын
In Azure Synapse Analytics, can we write back to the database using power apps from power bi dashboard.
@powerbiprodesigner79492 жыл бұрын
Hi. What I have read is that Azure Synapse Analytics (the MPP bit) has big advantages once the data reaches ~ 1tb in size. Else it is not cost efficient. Is this true ?
@kellimckee91212 жыл бұрын
Is Azure Synapse Analytics secure for GCC? My agency has been waiting for a while. 😉
@josuelabaut99692 жыл бұрын
can somebody recommend me a resource to learn python oriented to ETL/ELT data manipulation in notebooks? (synapse studio)
@FajardoCtba2 жыл бұрын
How it compares to Snowflake?
@ericmclaren4632 жыл бұрын
snowflake is better .. move cost effective.
@jessegorteronazure72242 жыл бұрын
whats your vision on large datamodels that do not fit in power bi? direct query to synapse sometimes sends strange sql queries that don't perform either on large datasets +1 billion rows
@fr1sket3632 жыл бұрын
Thanks for the video, Adam your mic sounds like it's clipping a little 😅.
@vikastangudu7125 ай бұрын
Energy is awesome!
@davidriffey65002 жыл бұрын
Could you use Synapse for a asset management database in the Microsoft environment or is there something else?
@McTwistedTwisties2 жыл бұрын
How does this differ from Azure Data Factory and can you have ingress or egress with it?
@anisnaim49472 жыл бұрын
Hello Will you more video related to this topic and maybe DP-500 ?
@jakobs23472 жыл бұрын
I think it is strawberries…
@jayopachecoea7 ай бұрын
Ayer vi un vídeo de ustedes, y hoy los busqué para que me expliquen Azure Synapse Analytics, y seguiré así porque explican bien. ¡Suscrito!
@bunty25772 жыл бұрын
I like to see a use case of predictive analytics that gives me insights on how to get started on this tool and bring the product to finish line... from raw data to end result in power bi
@iiiiii-w8h2 жыл бұрын
It's so frustrating to be subscribed to this channel. I only know some Power BI... and like what the fuck is spark? I really would like to understand the things you guys talk about on most of your videos but is way over my credentials.
@dezcraft_dev2 жыл бұрын
Hey Ivan. I was in the same boat as you a few years back. Guy In a Cube is great because they have LOTS of content for beginners (look at their "Getting Started with Power BI" playlist). But I promise if you keep watching the new content and do some google searching on the side as you go, you will eventually see how powerful each one of their videos is. From one PBI guy to another
@KaterynaYakubovska2 жыл бұрын
Great video! Want to know more about Synapse!!
@Baldur10052 жыл бұрын
I guess the problem (and oportunity) with Azure BI platform is that cost is often variable which discourage learning and experimentation from power users - something similar to Power BI Premium would be great which would lead to greater tool adoption
@kennylee18622 жыл бұрын
How do you migrate from an on prem sql data warehouse to Synapse?
@ameersuhelkhan43982 жыл бұрын
Loved the new intro and UI🔥🔥 #Bananas!
@Shadeth1352 жыл бұрын
Subbed in the hopes of synapse analytics content! Keen to see it in action, the whole process from ingestion to transformation to power bi. Would love some demos, touching on the main functionalities.
@CRDPARTH2 жыл бұрын
You guys are amazing !! Crisp information.
@PicaPauDiablo12 жыл бұрын
Synapse Analytics is definitely amazing. I was thinking about a good way to visualize Delta changes . B/c of all the fields and multiple versions, there's quite a bit of possibility.
@joeseroski10952 жыл бұрын
Love this. This is the content I'm looking for as I take the next step in my Power BI learning and my data journey.
@FajardoCtba2 жыл бұрын
How can I compare Synapse Analytics to AAS and SSAS. Can I use DAX? Does it has a visual diagram? It leverages column store? Does it has modern data sources with M code? How it performs?
@hugofernandohr2 жыл бұрын
I have the same doubt
@akthar32 жыл бұрын
is azure synapse as good (comfortable to use) as sql server query analyser/management studio ? or not yet ?
@ericmclaren4632 жыл бұрын
No.
@rafalg85752 жыл бұрын
Keen to learn more 😀
@whowho27458 ай бұрын
why "yo..."?
@GuyInACube8 ай бұрын
Why not?
@whowho27458 ай бұрын
@@GuyInACube it is lame.
@matthiask46022 жыл бұрын
I like the term "power house" . I would love to see a small usecase, so I can grasp the usefulness for my data
@sbdyelse2 жыл бұрын
Enough of all this talking , you know what we like you to do … so why not doing it this time ? Huh 🤔