Azure Stream Analytics Tutorial | Processing stream data with SQL

[@sud05]

These videos are even better than the live demos in my Azure Cert courses by Microsoft approved trainers 😁

[@AdamMarczakYT]

Wow, thanks! Appreciated :)

[@_indrid_cold_]

Adam, thank you so much for all your content. This is a fabulous e2e on IoT hubs and streaming. By far the best I have found; explain things incredibly well. MS should be recruiting you to do all their public training content!

[@AdamMarczakYT]

Wow, thanks! Glad you liked it.

[@arun06530]

That's a very crisp & clear video for Streaming Analytics in Azure.

[@AdamMarczakYT]

Fantastic, thanks!

[@user-bh3gf9pp3w]

Great video to show us this demo with multiple Azure service to do the IOT stream analytics! Thank you!

[@RahulDhanawade-im7su]

You are a saviour, i have my final round of interview and given similar task. Couldnt thank you enough for posting this! Thanks mate.

[@SandeepKumar-jw5gy]

Really you are amazing, ADAM. Your videos are mind-blowing, very easy to understand. Even the tough topic looks so easy after I watch your videos as your explanation and presentation are very clear.

[@AdamMarczakYT]

I appreciate that!

[@deepanshuaggarwal7042]

Hello Adam, your way of teaching is very good and your presentation enhances it more. Whenever I need to study something related to azure, first I check video from your channel. You are doing a great work. Thanks

[@sukheshchow6644]

Kudos to ur knowledge n presentation and making it a available for free cost🙏

[@AdamMarczakYT]

Thanks a lot 😊

[@sourabhsharma4878]

Very well demonstrated and step by step process. Keep up the good work 👍

[@AdamMarczakYT]

Thank you! Cheers!

[@DerickEhiobu]

Loved it, straight to the point. Learnt from this.

[@arvinbonggal5443]

I am preparing for an exam and got sick with MS learning materials this is much better from their sources, I hope they will structure all tutorial like this for easy understanding

[@AdamMarczakYT]

Thank you :)

[@TrivenSharma]

This is brilliant video for beginners.

[@AdamMarczakYT]

Great to hear, thanks!

[@dks82]

outstanding demo, Adam. Thanks for your easy teaching techniques to motivate towards technology learning.

[@AdamMarczakYT]

My pleasure!

[@YazminAbat]

Regret not having used these materials for passing az-900. Thanks a lot for such wonderful content!

[@ssrakeshsharma]

Too Good - Thank You So Much For Posting these Awesome Videos Guru Ji - Respect from India.

[@sathyamurthithangaraj9054]

yes this was an amazing , what we understand in 2-3 hrs , we can easily pickup in 30 mins , nice demonstration.

[@daywork1849]

man you make big smile on my face, Thanks

[@ramaseshaprasadgaduputi8029]

Excellent ! Thanks for helping in understanding the topics better and in a simple way.

[@AdamMarczakYT]

Great to hear!

[@gucciRouss]

Thanks Adam this video is very helpful, easier than the original one from MS. Thanks a lot

[@AdamMarczakYT]

Glad it was helpful Gustavo!

[@sboychev1]

As always, very good presentation and explanation. Thank you Adam!

[@AdamMarczakYT]

Glad you liked it! Thanks as always! :)

[@riteshsharma344]

Thanks for the great video Adam I always refer your channel in my free time to explore the new things.

[@AdamMarczakYT]

Awesome, thank you!

[@alitswq]

Amazing Video! Thanks Adam.

[@AdamMarczakYT]

My pleasure! Thanks!

[@Khevyn360]

Wow u just explaned me what was trying to undestand about 4 months!! Tks

[@purrinchi]

Best azure videos ever, and they are free ¡! Thank you a lot my friend

[@AdamMarczakYT]

Glad you like them!

[@ssunil1310]

Thanks for putting amazing content, its so easy to learn from your video..

[@charuchevy]

This was very well explained , Thank you Adam .

[@AdamMarczakYT]

Glad you enjoyed it!

[@SagnikSaha94]

Hey Adam, your videos are really awesome and informative. Thank you for creating such a resourceful videos.

[@AdamMarczakYT]

My pleasure!

[@alexandrefg8922]

Very instructive video! Thank you!

[@ngophuthanh]

Excellent video. Thank you, Adam.

[@AdamMarczakYT]

My pleasure!

[@rizomarshal7483]

this is just amazing walk-through. like all your content deeply explained with practical tips. learn a lot from this one - and was pleasure to watch. thanks for sharing your knowledge

[@johng5295]

Thanks in a million Adam! Very well explained. This is the best thing since slice bread + Power BI at a click of a button. This is the nth time that I am watching this again. Great content. Awesome. I couldn't find this explanation--simply put anywhere else. “Great teachers are hard to find”. Grade: A++ 💥

[@CoyoteMTB77]

Great explanation and clear content. Coming from a Microsoft guy too ❤️

[@AdamMarczakYT]

Much appreciated! Welcome aboard! :)

[@amitvijay84]

A very very nice tutorial. Thanks!

[@1277marina1277]

Super helpful video. Excellent job Adam. Please do more advanced video on this subject please

[@polinoma931]

Thank you Adam. Again great tutorial :)

[@HauNguyen-hr7qh]

Thank you for creating such awesome and informative videos. I really grateful for those!

[@MaKSokami]

Great video. Wish it covered needed accounts and roles to add inputs/outputs, in case the implementer is not an admin or owner of the input subscription

[@friendofjesus5863]

Awesome Adam,stay blessed

[@muhammadrahman3510]

Excellent explanation!!!

[@ravikumar-pw9kp]

Nice Presentation, Thank you!!

[@arulmouzhiezhilarasan8518]

Fantastic demo and explanations Adam! Kudos!

[@AdamMarczakYT]

As always glad you enjoy it :)

[@prasadmadushan]

To be honest, I can understand this easily more than Microsoft doc

[@AdamMarczakYT]

Awesome, thanks! :D

[@c.4800]

Congratulations! Great job!

[@AdamMarczakYT]

Thank you so much 😀

[@Search4Knowledge]

This was a GREAT demo!

[@AdamMarczakYT]

My pleasure!

[@rajkumarjegadeesan9792]

Very good presentation on E2E scenarios. Thanks.

[@AdamMarczakYT]

Thanks! :)

[@taguada3024]

You always give us a good overview for the proposal of videos. Congratulations :)

[@AdamMarczakYT]

Thank you very much!

[@williamjavierjimenez9451]

excellent video friend, thanks for sharing your knowledge, greetings

[@AdamMarczakYT]

Thank you! Cheers!

[@ramt179]

This is simply great way of presenting. Thank you!

[@AdamMarczakYT]

Glad you like it, thanks for stopping by :)

[@kothiyaVasahat]

Good explanation given Adam, it helped me while I was looking for streaming analytics videos !! Thanks for making

[@AdamMarczakYT]

My pleasure!

[@ferpeo]

Really well explained! I'm sure it took you a lot of time to do this video and I Thank you very much for it! It will help me out with my Master Thesis

[@AdamMarczakYT]

Awesome! Thanks :)

[@guillaumemichael951]

Awesome! Explanations are clear and complete, thanks

[@AdamMarczakYT]

Glad it helped!

[@harpreet_singh_walia_youtube]

Thanks a lot, very nice presentation

[@memetank.]

Dude. You are fantastic

[@matthewmark7224]

this is amazing. thank you.

[@user-uw8kq9bh1j]

Great content Adam.

[@viveksagarpandey6627]

Loved it!! Thanks a Lot 😊

[@andreaskurz6166]

Just great tutorial. Brilliant work!

[@AdamMarczakYT]

Thank you! Cheers!

[@Nikki24262]

This is amazing, Adam :)

[@AdamMarczakYT]

Thank you! :)

[@thenayeem1]

Hi Adam, Can we use multiple device for the same IoT Hub. I was facing issue while using multiple devices in the same IoT hub. It is picking only from one device and ignoring other.

[@AdamMarczakYT]

Of'course you can. It wouldn't be an IoT 'hub' if you would be able connect multiple devices. You might have made mistake somewhere else.

[@michalhutny7356]

Great end2end scenario, especially your IoT simulator! :D However, I'm suprised that You didn't show other windowing functions like session, sliding and hopping.

[@AdamMarczakYT]

Thanks. It's always hard to choose which are the TOP features and which aren't. I mentioned window functions in additional features. But I wanted to show live dashboarding more.

[@brunexjundiai]

Awesome video, cheers

[@AI-Health-posts]

Fantastic video Adam. many thanks for that.

[@AdamMarczakYT]

Glad you enjoyed it

[@vaibhavbapat9090]

Nice

[@AdamMarczakYT]

Thanks

[@RonaldPostelmans]

Nice explanation. Ik hope it is still working the same now.

[@Jkprasad]

Simply superb you rock!!

[@AdamMarczakYT]

Thanks mate! :)

[@MsAlpha]

Excellent video well articulated

[@AdamMarczakYT]

Thank you kindly!

[@jpb2085]

Brilliant! Subbed!

[@rprasanth5235]

Very useful . Excellent job

[@AdamMarczakYT]

Thank you! Cheers!

[@venkatpunnam1655]

Precise and excellent...

[@AdamMarczakYT]

Thanks!

[@nehapathare3519]

Very helpful. Best video I found. Thankyou🙂

[@AdamMarczakYT]

You're welcome!

[@nehapathare3519]

@@AdamMarczakYT The blob storage output asks for necessary field minimum rows,hours,minutes unlike shown in your video. What values should I put there?

[@ajitheshgupta3017]

You are just amazing :)

[@jeffrey6124]

Great video Adam! 😍 Do you have a similar video but incorporating Python? Thanks

[@lokeshramaraj2800]

Amazing content. Thanks Adam

[@AdamMarczakYT]

Appreciate it!

[@younusmohammad6050]

Love You brother , what a demo

[@AdamMarczakYT]

Cheers!

[@BijouBakson]

Thank you

[@ritikeshsingh5940]

Thanks for this but it would be great if you can tell us how to deal with multiple sensors

[@AdamMarczakYT]

Well, multiple sensors just send their unique deviceid with the data. So just write appropriate SQL queries as per your business logic.

[@eirikandersen9848]

Hi Adam! Great video, as always:) I have one question regarding adding new inputs and outputs. When you stop the streaming job to add new input and output you loose data, right? If so, one should take time to design the solution so that you do not loose data. Additionally, there is no point in creating every output and input to safeguard against loss of data because of additional cost. I was just thinking out loud here :P

[@AdamMarczakYT]

Stream analytics has option to continue where it left of from the event hub as long as the event hub still has those messages (configured via retention period). :) For the loss of data event hubs has capture feature which by default dumps all the stream data into blob for long term backup/retention/analytics.

[@ashokms]

Well depicted end-2-end journey! Btw your video editing is too good, what tool do you use?

[@AdamMarczakYT]

Thanks! I use visio for diagram and camtasia for recoding :)

[@edalejo3061]

This is very informative. I am using your videos for me to get familiar with different tools in Azure. Just a quick question - in your first example (Blob as an output). Will the data in the blob double because you're selecting everything from iotinput and inserting in bloboutput without any conditions?

[@AdamMarczakYT]

Correct. With no aggregations/filters/selects you just make a dump of your stream data on the blob. Thanks for watching :)

[@xavierjohn9]

Spinning up a "Stream Analytics job" per query does not seem feasible. I wish there was a product where I can have several standing queries that can be maintained via SDK. Putting multiple queries in a single query window is hard to manage.

[@AdamMarczakYT]

I agree, although it is how it is. Maybe try different streaming technology that is more flexible and matches your needs. For example maybe use Azure Databricks?

[@xavierjohn9]

@@AdamMarczakYT I will take a look at Databricks. I wonder if Azure Synapse Analytics is an option if the storage is Cosmos DB.

[@n1gzd]

I have been watching a lot of your tutorials. You do a great job with your presentations. I have been looking for a specific playlist supporting DP-203 exam. I have not found one that I like yet. Are you planning on presenting one (I hope)?

[@vaasu]

Awesome Demo!!

[@AdamMarczakYT]

Glad you think so! Thanks!

[@georgezviadgoglodze7810]

Great work!

[@AdamMarczakYT]

Thank you! Cheers!

[@drteddyweb]

Nice video, but I have some questions 1. When we hit the refresh button the data are not actually refreshed and only the first 50 messages are shown everytime. 2. How about adding a second input (I have set two RaspBerry emulators that sending messages) and a second output also. The query does not work. Here is mine: SELECT * INTO powerBi12 FROM raspinput WHERE deviceId='Raspberry Pi Web Client' SELECT * INTO powerBi1 FROM raspinput WHERE deviceId='Raspberry Pi 2' When I hit "Test query" only the first SELECT statement works. "rapsinput" in the above query is an IoTHub with two devices. Thanks in advance

[@UMAMAHESHYADAVA]

Very good explanation Adam, Thank a lot .. How Testing can be done On azure Live analytics streaming data?

[@AdamMarczakYT]

Hi there, I usually just test on some 'dev' instance, but I haven't found good method/tool yet myself for that as I'm not working much with streaming data. Thanks for stopping by! :)

[@user-ul6tf3dp9v]

at 6:12 you say replace the string, yes but replace by what ?

[@shantanudeshmukh4390]

Nice Video Adam. Good to learn azure stream analytics. Is there any video for synapse analytics as well?

[@AdamMarczakYT]

Thanks, not yet, but I have it in plans!

[@shantanudeshmukh4390]

@@AdamMarczakYT Great. I am waiting for one !

[@krishabasrur383]

Hey Adam. Nice video. Could you please let me how I can process i.e transform, aggregate huge source data and load to synapse db? The transformation is really complex as there are multiple joins and transformation. The output of one table will be used as input to other till all intermediate data is joined and loaded to final reporting aggregated summary table.

[@lcanchedou2413]

Hi Adam, I know that EventHub has data retention period. It is possible to fetch data from a time window using stream analytics queries? Regards

[@Eubilecki]

Excellent Video again! One Question can I connect Event Hub on SAP ECC? something like we connect on Data Factory by Sap Table?

[@AdamMarczakYT]

Hey Raphael, thanks! I'm not an expert on SAP ECC but I understood that that it supports AQMP so potentially you can stream your data to Event Hubs. :)

[@chandrakanthkothapalli9434]

If the input is a json message how will that be parsed/flattened by the stream analytics when writing on to blob to use it on Power BI service ? in this scenario can the stream analytics supports writing some java code to flattened the json message ?

[@gizemengin1550]

Dear Adam, how I can ingest data in Iot Hubs or Event Hubs with API Key? I study for CoinMarketCap data but I could not transfer the data to Event Hubs.

[@anshulanilgaur8365]

Awesome video! But I have one doubt regarding this, what if someone wants to process stream data from various different IoT sensors at the same time (simultaneously), then can different Output Folders (with each IoT sensor's stream data in Excel files) be created for each IoT sensor's stream data? Could you help me with this, thank you so much!

[@AdamMarczakYT]

Well that's already a very specific scenario, as a data engineer you need to be able to put all the pieces together. To work with excel files and streams Azure Databricks might be a good choice.

[@roshangangurde7188]

At 4:03 in 2nd tumbling window output values should be 1 right? Can anyone please explain me why we are getting 2 there.

[@mohammedalidinar8570]

Hi Adam your tutorial is absolutely amazing its help me to finish my Hons Project, but I am really struggling with something not relating to this but its within the data so I have integrated my data from pybytes but the thing is all my sensor reading comes as signals and I want to sort this sensor reading as I have three different reading and won't allow me to show them organised so I am struggling on that bit is there way to sort this signals coming from pybytes to the reading names I have three reading Temp and Acce and Humidity

[@sarahlares1249]

I love this channel

[@AdamMarczakYT]

Thank you :)

[@crazybauns]

wont your power bi data set explode in size after a while?

[@AdamMarczakYT]

PBI Streaming Datasets have row limits but also only store certain amount of data, feel free to check this article to understand the limitations docs.microsoft.com/en-us/power-bi/connect-data/service-real-time-streaming?WT.mc_id=AZ-MVP-5003556#streaming-dataset-matrix in case of this demo 200,000 rows are stored docs.microsoft.com/en-us/power-bi/connect-data/service-real-time-streaming?WT.mc_id=AZ-MVP-5003556#using-azure-stream-analytics-to-push-data

[@olenadenysiuk5332]

Hello Adam, did you create two outputs with the same dataset and different table names? Did it work?

[@AdamMarczakYT]

Not sure I understand but single source can output to multiple outputs as shown in the video.

[@TheNishi42]

Any video on azure anomaly detector for live streaming data on azure ?

[@AdamMarczakYT]

Not yet :(

[@alokverma7327]

Hi, Thanks. This is an amazing video for a beginner like us. One query: What needs to change in the python program if we will use a real sensor such as ADXL345 with raspberry pi instead of using simulator? Kindly revert soon or let me know if anyone can answer.

[@AdamMarczakYT]

Thanks. If you use your own device then check out Microsoft docs for IoT hub or Event Hub and write code for that device to send telemetry data. Here is an example in C docs.microsoft.com/en-us/azure/iot-hub/quickstart-send-telemetry-c?WT.mc_id=AZ-MVP-5003556 but you need to check tutorials for your own device and supported languages.