No video

Azure Data Factory Custom Email Notifications Tutorial

  Рет қаралды 84,352

Adam Marczak - Azure for Everyone

Adam Marczak - Azure for Everyone

Күн бұрын

Пікірлер: 183
@jeffti5220
@jeffti5220 4 жыл бұрын
Thanks!!! You are the best! Keep producing these videos. A future suggestion - a video applying SCD techniques in ADF
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Thanks Jeff. SCD is surely a good topic, I'll try to get that video somehow on the schedule :) Thanks for watching!
@mukundakkoor4799
@mukundakkoor4799 3 жыл бұрын
Adam, This is an awesome tutorial to learn sending emails from ADF. Thank you.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Glad you enjoyed it!
@MsEdward1990
@MsEdward1990 3 жыл бұрын
Thank you for the great example of how to use logic app and ADF.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
My pleasure!
@Navi24
@Navi24 3 жыл бұрын
Thanks Adam, this video was a blessing in disguise. Thanks a-lot for posting this video.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Thanks! I hope I don't disguise my videos too much, I want them to be inviting ;)
@mukundakkoor4799
@mukundakkoor4799 3 жыл бұрын
Adam, Excellent tutorial. Very precise and shown clearly with examples. Thank you.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Glad it was helpful!
@fernandodelago
@fernandodelago 2 жыл бұрын
Hi Adam. Your video have a excellent explanation. This help me a lot. Don't stop! Continue and go ahead! Congrats!
@yemshivakumar
@yemshivakumar 4 жыл бұрын
All your videos are crystal clear concept explanation and fantastic demo videos including this.
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Thank you, appreciated! :)
@NehaJain-ry9sr
@NehaJain-ry9sr 4 жыл бұрын
My favourite channel to learn Azure.. Thanks Adam for awesome videos on Azure .. keep it up.. my best wishes to you :)
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Thanks a ton!
@murugans5540
@murugans5540 2 жыл бұрын
Very much useful topic and good hands-on eyeopening experience
@kannanpadmanabhan6681
@kannanpadmanabhan6681 3 жыл бұрын
Awesome video. I love it. Thanks for the tutorial. I can't wait to test this.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Glad it was helpful! Thanks :)
@watarusuzuki6366
@watarusuzuki6366 3 жыл бұрын
Wonderful work Adam! This is exactly what I was looking for.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Perfect! Thanks for stopping by!
@prakashloganathan5726
@prakashloganathan5726 3 жыл бұрын
Hi Adam. Awesome content with real-time use case 👏
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Glad you liked it! Thank you :)
@AshokKumar-ji3cs
@AshokKumar-ji3cs 3 жыл бұрын
Hi Adam, thanks for the video it has valuable information. The way how you illustrated really awesome. I really appreciate your efforts. Request you to do more videos on data Engineer 🙏🙏
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Awesome, thank you!
@tamitang8731
@tamitang8731 3 жыл бұрын
Hi Adam, this is a super clear and helpful tutorial. Thanks for posting it!
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
You're very welcome!
@GiovanniOrlandoi7
@GiovanniOrlandoi7 Жыл бұрын
Great video! Thanks Adam
@cavalca
@cavalca Жыл бұрын
Your video helped me a lot. Thank you very much.
@vaddiparthivenkatamuralikr8334
@vaddiparthivenkatamuralikr8334 3 жыл бұрын
Excellent demo
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Cheers! :D
@dashingdaring
@dashingdaring 2 жыл бұрын
Excellent Video Adam. Thanks
@NeumsFor9
@NeumsFor9 3 жыл бұрын
Regarding parent child handling of email success/failure subtly makes me appreciate event handlers in SSIS but also the parent-child package reusability story as well......without the complexity of bubble up behavior and scopes, of course!
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
I agree. Flow-control is something I definitely hope that MS will expand on in the future. Same with no 'terminate' block for 'throw error' scenarios. ;)
@sravanidevi8785
@sravanidevi8785 4 жыл бұрын
Hey Adam.. Thanx a lot for such a cool and clear explanation!!
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
My pleasure!
@raphaelboateng-amponsah1363
@raphaelboateng-amponsah1363 4 жыл бұрын
Really appreciate your videos, such good quality!!
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
I appreciate that!
@AlexandreShimono
@AlexandreShimono 3 жыл бұрын
Awesome video! Relevant, well designed. Wish there were more content like yours on youtube and less Flat Earthers! Cheers from Brazil.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Much appreciated!
@chinny4953
@chinny4953 3 жыл бұрын
Excellent video man. Great work. Thank you.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Much appreciated!
@chris4321das
@chris4321das 4 жыл бұрын
Excellent stuff, Adam! Invaluable, better than online courses I paid for (tylko mow 'debug' jak 'dibag' =) ).
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Wow, thanks! No czasami mi sie zapomni a potem do konca video zle mowie :D
@chris4321das
@chris4321das 4 жыл бұрын
@@AdamMarczakYT Super mowisz, wyraznie.
@Info-Galaxy
@Info-Galaxy 2 жыл бұрын
Thank you Adam for such a crystal clear demo. It really helped me
@dancrowell2933
@dancrowell2933 4 жыл бұрын
Hi Adam, your tutorials are excellent. Can you do a tutorial on using Git and moving ADF code from development, to test, to production?
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Definitely a topic on the list. It's a bit complicated topic so I need to figure it out couple of items, but I hope to get around to this topic soon. :) Thanks for stopping by!
@shivanidubey1616
@shivanidubey1616 3 жыл бұрын
Hi Adam can u pls explain how we perform version controling of adf piplines and how to deploy azure data factory pipline into dev uat and prod
@BlackJackFilm
@BlackJackFilm 3 жыл бұрын
@@AdamMarczakYT We are now trying to figure that one with DevOps... hard piece of cake. Greets from Poland ;-)
@whynot19001
@whynot19001 3 жыл бұрын
Thank for this my friend!!
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
No problem 👍
@perukasr
@perukasr 2 жыл бұрын
Thanks! I used the same logic in azure synapse but had issue to capture dynamic error message from data flow. The fix for this is : "@{ replace(activity('Execute Pipeline1').error.Message,'"' ,'' ) } ",
@gbw5679
@gbw5679 4 жыл бұрын
Well done! Thanks Adam for the good walk-through exercise :)
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
My pleasure!
@sangeet051
@sangeet051 3 жыл бұрын
Excellent!! Clear and precise
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Glad you think so!
@arulmouzhiezhilarasan8518
@arulmouzhiezhilarasan8518 3 жыл бұрын
Much informative, thanks Adam!
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
My pleasure!
@kevalvankudre
@kevalvankudre 2 жыл бұрын
Great tutorial !
@NehaKumari-su1gg
@NehaKumari-su1gg 3 жыл бұрын
Your videos are very helpful and your way of Presentation and explaining each and every topic make everything so simple . Thank you Adam this is what I was looking for on a starting level. One question can we get the pipeline start and end time in any way in ADF?
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Thank you so much 🙂
@gaurimorajkar3234
@gaurimorajkar3234 3 жыл бұрын
Cannot be better than this
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Great to hear!
@SanjiVSwaraJ
@SanjiVSwaraJ 3 жыл бұрын
Excellent tutorial..
@CoopmanGreg
@CoopmanGreg 3 жыл бұрын
Excellent!
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Thank you! Cheers!
@balanm8570
@balanm8570 4 жыл бұрын
Awesome video as always !!!
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Thanks again!
@frenamakenson9844
@frenamakenson9844 11 күн бұрын
Dam ! I'm from the Power PLatform World, Nice demo One thing, One the Technical Account on Logic App, PassWord never expire should be configured to maintain consistensy on the Outlook connection.
@dariom5073
@dariom5073 3 жыл бұрын
Thanks Adam!
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
My pleasure!
@vipinpanikar1
@vipinpanikar1 3 жыл бұрын
Hello Adam, thank you for making these videos. One help. Could you please make another video about how to attach attachments to these notification emails. These attachment could be again a document from the blob storage.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
You can use Logic Apps blob connector to get file content and add it as attachment to an email.
@RC-nn1ld
@RC-nn1ld 4 жыл бұрын
seriously good video!
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Thank you Roger!
@anonpheles
@anonpheles 3 жыл бұрын
Thanks, Adam! Great tutorial. What you recommend for outgoing notification emails? Office 365 service account or SMTP Relay service?
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Sometimes it's more of a question what is your company policy on that. In many enterprises SMTP is a way to go, I typically use the one that works best for my scenario. If I have smtp relay I use that, because o365 account requires exchange license.
@ofeliajesus2385
@ofeliajesus2385 Жыл бұрын
Great video ... One question is it possible just one email for several pipelines I would like to have daily an email with the pipeline's last status failure or success ??
@dianamilenacoronadogranado2259
@dianamilenacoronadogranado2259 3 жыл бұрын
Hi adam!! thanks a lot for yout tutorial!! I'm having a problem..I have got the 2 pipelines, however when a run the DEMO-PIPELINE, the Master(email pipeline) it's no executed , I follow all your tutorial but I'm not able to do it...The email pipeline is invoking the DEMO-PIPELINE should I do something else? or add another thing on the DEMO-PIPELINE dashboard?
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Trigger master pipeline, not the demo-pipeline. Parent pipeline won't be called if you call demo-pipeline. That's why it's called master.
@davissonf.coelho1003
@davissonf.coelho1003 3 жыл бұрын
Good afternoon, your explanation was fantastic, but I think it is insane not to have a direct component for shooting. If you have the programming of only one trigger, could you tell me? thankful
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Direct component for shooting? Not sure what do you mean by that :) But thanks for watching, glad you enjoyed the video. Also "If you have the programming of only one trigger, could you tell me? thankful" not sure I understand this question either. Can you please rephrase it?
@martinbobak622
@martinbobak622 3 жыл бұрын
Hi Adam, Great video! If I wanted to send a different email (different body of the email) for different pipelines, would I need more than one logic app?
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Or you can parametrize the body and construct it in ADF, but be careful about this so that noone will impersonate you :) For safety reason I wouldn't do that.
@msummerma1
@msummerma1 2 жыл бұрын
Hi Adam, I tried to reproduce your scenario. Unfortunately I encountered Problems with the following sentence: "message": "@{activity('POC Alerting Failure').error.message}". The activity('POC Alerting Failure') is a pipeline but the mail won't be send. When i replace the pipeline with another activity like a Lookup (which produces a failure) then the mail will be send correctly. Do you maybe know what has changed here?
@shaikmussarath8301
@shaikmussarath8301 2 жыл бұрын
Hi Mark, Been following your videos. Thank you for the rich content. Just having a doubt about notifications. In Email From address your name can be seen. Is there a concept of Service Account to configure the From Email address just like SQL Server Agent so that we can avoid showing your individual email address. Much appreciated.
@yaroslavfomenko2803
@yaroslavfomenko2803 3 жыл бұрын
Thank you for your video! May I ask a question, you said it wasn't secure to send the recipient as a part of the JSON body. But isn't the HTTPS body by definition encrypted? Would a malefactor still be able to see the contents of the request?
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
It's encrypted but if you use Logic Apps like this then it's a publicly available URL which if exposed can be used by anyone, anywhere in the world. So if someone would get it they would be able to use that URL to send emails to anyone they want. Sometimes it could be employee who commits code to a public repo by mistake, etc. Things happen. It's a low risk thing but why take it?
@yaroslavfomenko2803
@yaroslavfomenko2803 3 жыл бұрын
@@AdamMarczakYT Thanks for the quick response! I see what you mean. But the malefactor would be able to use the app to send emails only if (s)he knows the JSON structure, that there is a field called "recipient". Maybe it's wise to use code names for the fields instead.
@alexrusnak7574
@alexrusnak7574 4 жыл бұрын
Cool stuff! One question - how about instead of creating a wrapping pipeline for error reporting, chain error email action on success email action with dependency condition 'skipped'?
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
This potentially could work but it would be tricky to figure out which block raised error and would require custom logic to get that error message. it would increase the effort required when modifying pipelines and could result in missed error messages, especially if you would have many parallel branches in one pipeline. But for simple cases you potentially could do it.
@user-cu4ir1qi6w
@user-cu4ir1qi6w Жыл бұрын
How do you send email using outlook if you have a microsoft exchange outlook account, assuming office 365 is saying rest api is not enabled
@praveensharma-uw8ds
@praveensharma-uw8ds 2 жыл бұрын
thank u for sharin
@marcing8325
@marcing8325 3 жыл бұрын
Hi, I tried this and I am getting an error : 'After parsing a value an unexpected character was encountered: \" . When I look ate input I see \ before every ":" . Any idea what may be the reason ?
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Hey Marcin. I’ve ran this demo like 10 times, so I’m very positive it works. That means you’ve made a mistake somewhere. My advise, its best just to do last few steps over, sometimes it’s easier to find a mistake this way.
@dharmveerchaudhary7060
@dharmveerchaudhary7060 2 жыл бұрын
Hi ADM , how can send failed adf notification to multiple mail or any DL or ALL mail id comes in any resource group
@bahadrguler2404
@bahadrguler2404 2 жыл бұрын
Hey Adam thanks for great content. My problem is that I want to compare two columns value in azure postgresql and send email notification if there is any extraordinary difference between them mathematically. Which tool should I use on azure, I am quiet confused. Any help would be appreciated. Thanks in advance.
@naveshkumar
@naveshkumar 2 жыл бұрын
interested in this
@satyaraj1427
@satyaraj1427 3 жыл бұрын
Hey Adam....There was an error while logging into the outlook account...The error states as follows...".Failed with error: 'The browser is closed.'. Please sign in again." can you help me with this... i tried doing with my corporate mail also but it is not happening
@tanupriya9849
@tanupriya9849 2 жыл бұрын
Hi Adam, Thanks for such an informative video. Could you please suggest how can we configure to send an email if any pipeline gets stuck/ queued or long running?Thanks in advance!
@pushpanthkumar9028
@pushpanthkumar9028 Жыл бұрын
Hi priya..May I know how did you solved this problem please..
@sachinborkar416
@sachinborkar416 3 жыл бұрын
Excellent one, thanks very much. Do you have any video on reading multiple csv filenames from Azure blob storage to databricks using pyspark or python.?
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Check out my databricks tutorial. I have few samples there using Python and Scala. The difference between python and scala for basic tasks is so minimal that you can easily grab my Scala scripts and with minor changes use them for python.
@SuperJamu
@SuperJamu 2 жыл бұрын
Hi. Is an outlook email required to do this? Can i use my corporate gmail to send the message?
@ericrouach
@ericrouach 3 жыл бұрын
Hi Adam, could you please explain how to add a file processed in the pipeline as an attachment to the email? Thanks a lot! Great tutorials!
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
You probably would need to save output to blob storage and pass the blob path to a logic app and in logic app read the file using path and add it as attachment.
@ericrouach
@ericrouach 3 жыл бұрын
@@AdamMarczakYT Thanks Adam for your quick reply! I appreciate! I am keeping the file in a blob storage. I'm just struggling trying to pass the blob storage path as a parameter in the web activity settings body section. Is it possible to add an "Attachment" element to the JSON?
@viswanadhsai5963
@viswanadhsai5963 18 күн бұрын
Hi I'm getting this error, Invoking endpoint failed with HttpStatusCode - '404 : NotFound', message - 'The requested endpoint(url) does not exist on the server. Please verify the request server and retry.' You guidance could be a great help
@viswanadhsai5963
@viswanadhsai5963 18 күн бұрын
While triggering logic app server
@BijouBakson
@BijouBakson 2 жыл бұрын
Thank you
@gautam26trivedi
@gautam26trivedi Жыл бұрын
That was a great knowledge share. Can you give me some ideas about from datafactory source as Azure SQL & Sink as an excel file. uploading to sharepoint. How to achieve this???
@gauravjoshi5054
@gauravjoshi5054 3 жыл бұрын
Simply awesome thanks a ton Adam. can you please help me with how to send email notification in Azure devops release pipeline if one stage is failing like i have QA , DEV , DEV1 stages i want to send email on if any of these stages failed for any reason. I don't see any inbuild option in Azure for specific stages. Your guidance will really help me.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Thanks! Check your notification settings because normally you should get emails when pipeline fails docs.microsoft.com/en-us/azure/devops/notifications/manage-team-group-global-organization-notifications?view=azure-devops?WT.mc_id=AZ-MVP-5003556
@haivercarrillo8603
@haivercarrillo8603 4 жыл бұрын
Hey Adam as usual, great work, quick question, do you know any tool to perform database sharding using Azure and not using Cloud Services?
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Hi thanks! Not sure I understand what do you mean using Azure but not cloud services.
@haivercarrillo8603
@haivercarrillo8603 4 жыл бұрын
@@AdamMarczakYT right, didn't clarify that, I meant the Cloud services PaaS docs.microsoft.com/en-us/azure/cloud-services/cloud-services-choose-me I've seen that I can only use this tool to perform sharding (split-merge) but that's going to deprecate in the near future.
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Where did you find the info that this will be deprecated? Nowhere in the documentation it says so.
@CodeCraftWithVinayak
@CodeCraftWithVinayak 4 жыл бұрын
Hey Adam, can we write integration test cases for logic app and pipeline if yes plz can u share something, thanks in advance.
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Not that I'm aware of at this time :( Good question though!
@ganeshvishnu1171
@ganeshvishnu1171 3 жыл бұрын
Hi, I need to add recipient email address dynamically. May be more than one. But not hard coded, using a csv file or from a table, i need to pass to the parameter. Can you help me in this ?
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
You can use Lookup action and pass that output to Logic Apps as one of the parameters.
@pedro9485
@pedro9485 4 жыл бұрын
Super! What about this scenario - I am importing some data. Imagine there is a Column with name "Running". The row is True or False. How to make alert via mail in case the column "Running" will be False? Is there any possibility how to make it? Thank you, Adam !
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
I'd probably try using Lookup to source data and create foreach with conditions to send emails. Note that lookup should be used for small datasets. Everything depends on details! Thanks for watching!
@avnish.dixit_
@avnish.dixit_ 3 жыл бұрын
Hi Awesome Adam.. I tried using @activity("activity_name").error.message to send the error message but I gives an error that "The request content is not valid and could not be deserialized.. After parsing a value and unexpected character was encountered:.. Path 'message'
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Hard to say without looking at this. Maybe you've made a mistake somewhere. Check in the debug the output of that activity to verify if objects for error and message are there. Sometimes it's faster just to delete this and try again, it's also good for learning. :)
@avnish.dixit_
@avnish.dixit_ 3 жыл бұрын
@@AdamMarczakYT thanks I'll check again for now I hard coded it
@joeklingaman1032
@joeklingaman1032 3 жыл бұрын
@@avnish.dixit_ Please try pasting your body content into the body text field box WITHOUT clicking on 'Add Dynamic Content' in web activity calling Logic App
@piyushppatra
@piyushppatra 3 жыл бұрын
Hello Adam, this is very helpful. But I am getting error when I the previous activity error message syntax.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Make sure to pass proper JSON to Web Activity like {"Message":"@activity('Execute Pipeline').error.message"}
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Remember that with every tutorial I provide samples on github. Link is in the comments so you can quickly copy and paste the templates and request body.
@hemanthsai7465
@hemanthsai7465 3 ай бұрын
Can i use functions instead of logic apps?
@abinandhp9190
@abinandhp9190 3 жыл бұрын
Hi Adam, your tutorials are excellent. Can you guide me on how to connect ADF and Snowflake? I would like to copy flat files from Azure Blob to dynamic tables in Snowflake.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Hey thanks! Did you try using linked service for snowflake? It should work without any problems.
@abinandhp9190
@abinandhp9190 3 жыл бұрын
@@AdamMarczakYT I tried the linked service for Snowflake. By creating the SAS token URL for the blob storage, I established connection between snowflake and Azure Blob storage. However the CSV files read using copy activity couldn't be converted to dynamic tables in snowflake!!. Only when the appropriate table pre-exists in snowflake, the files get copied. Is there a way to create dynamic tables for the appropriate files.
@Magelan913
@Magelan913 4 жыл бұрын
Hi Adam, thank you this tutorial, can we add attachment to email, such as file or query result..?
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
You can, logic apps allow for attaching files to emails. Check the docs for details :) Although if query result would be big I'd probably save it as a csv on blob and just generate link to it from logic app and attach that in the email. Small query results could be send directly to logic app in the body.
@ambujineebisoi9860
@ambujineebisoi9860 3 жыл бұрын
Thank you Adam for this grt video. I have tested this whole video but facing one issue. The color coding is not working in the email body.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Not sure, it worked for me last time I checked. Please check the video and ensure all stops in the logic app were done the same way. Especially passing HTML body as variable. If it won't work let me know.
@ambujineebisoi9860
@ambujineebisoi9860 3 жыл бұрын
@@AdamMarczakYT Thank you.Now it's working for me.Just added a semicolon after the color and it started working.
@jagadeeshpinninti3323
@jagadeeshpinninti3323 2 жыл бұрын
How much will it cost to use using Azure logic apps to send the failure ADF pipelines email notifications?
@dominicwrapson1199
@dominicwrapson1199 2 жыл бұрын
We had this Logic App approach configured several months ago by a contractor, with just 3 ADF pipelines running once per day (so 3 total per day), only sending on failures which was only couple per month. We have 3 subscriptions for our different deployment levels (dev/test/prod) and the Logic Apps purely from the service charges (App Service Plan) have cost us $500USD/mo. The pricing is a little misleading as the calls/actions themselves cost _very_ little, but just to have the app exist and listen costs quite a lot.
@mainuser98
@mainuser98 5 ай бұрын
Can we do this with azure alerts?
@chen5576
@chen5576 4 жыл бұрын
This is really informative. May I please ask if there is a way to send a text file to an office 365 account as an attachment? Could we add a trigger on that as well? Let's say if the txt file updates then send the attachment, else do nothing? Thank you!
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Hey Chen. Yes you can. Check this guide, it shows briefly how to add attachment to email via logic app.
@chen5576
@chen5576 4 жыл бұрын
@@AdamMarczakYT Thanks Adam, which guide you are referring to?
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Uhhh link is missing, here is the guide: docs.microsoft.com/en-us/azure/logic-apps/tutorial-process-email-attachments-workflow :)
@kirubababu9255
@kirubababu9255 2 жыл бұрын
How to send outlook email without 'From' from logic app?
@sandeep5996
@sandeep5996 4 жыл бұрын
Nice work Adam! Could you please make a video on. How to check 0Kb csv files / zero row record from source. If zero Kb file/Zero records in source trigger an email, in azure Data Factory. Thanks in advance.
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Hi Sandeep, if you are aiming to be a data engineer then those will be your every day challenges that you will need to solve. Typical data engineer should understand the technology, be able to search documentation online, explore, play around with it, and finally use those skills to combine and build working solution. I can't create video on every single scenario. You already stated what needs to be done, split it into separate items and search in google on how to get them. 1. how to check file size with ADF > will probably return that you need to use get metadata action 2. how to create if in ADF > will point you to conditions 3. my video on bulk load will show how to iterate 4. my video on emails shows how to send email just combine this and build the solution. You got this.
@sandeep5996
@sandeep5996 4 жыл бұрын
@@AdamMarczakYT thank you Adam. For step by step explanation. Happy to let you know, I sorted my posted query.
@samorodko
@samorodko 2 жыл бұрын
Am I the only one struggling because Microsoft has changed the Logic Apps UI? First of all, I needed to create a workflow which wasn't the case for Adam. Now I am struggling to find rub history. The pipeline debug was successful, but workflow Run History is completely empty...
@sathishreddy2168
@sathishreddy2168 2 жыл бұрын
Is it possible to set up email alert for any pipeline failure in ADF?
@Info-Galaxy
@Info-Galaxy 2 жыл бұрын
Yes satish, it can be done, jump on 21:14 in this video for more details
@srinivasreddymalem6494
@srinivasreddymalem6494 Жыл бұрын
@Adam, My question is how to send a notification when a pipeline/pipelines execution duration is >10 mins under a data factory. Much appreciated your help.
@akshatanand8027
@akshatanand8027 3 жыл бұрын
Hey, I want to perform delete activity in adf but first want to ask user permission. Like if user will get any email or alert that pipeline has been failed so do you want to delete all data? And if userr opt for yes than delete activity will be performed. Can you help? I hav tried to use "send email with option in logic app" but not working. I want my pipeline to run on the basis of that answers. If yes than delete If no than dont delete.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
You can either use webhook activity in ADF or start new pipeline. For approval process you can use logic app with action called send approval email.
@akshatanand8027
@akshatanand8027 3 жыл бұрын
@@AdamMarczakYT yup working , thanks man!!
@reddyvemula1393
@reddyvemula1393 3 жыл бұрын
Hi Adam,How to send email with attachment(blob storage file)
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Send email allows sending attachment. Check some blogs/docs for examples stackoverflow.com/questions/51473878/azure-logic-app-how-to-send-an-email-with-one-or-more-attachments-after-gettin
@hemanthkh
@hemanthkh Жыл бұрын
how to cover the error messages in alerts
@santhoshreddykesavareddy1078
@santhoshreddykesavareddy1078 3 жыл бұрын
Thanks much for the video Adam. I have implemented this in my solution. And I took it little advance by passing the values to the elements by using a lookup activity. Excel file ( which has column names as Jason element name and value ) For example : "Title": "@{activity('Lookup1').output.FirstRow.Title}" This way I can modify the values outside the pipeline without touching the code But the problem is.. in my excel the title value is : Failed at @{utcnow()} and the adf is not converting that into time , it is populating as is to the email. My email body is looking like this Failed at @{utcnow()} how to print the formulae value in the email Please help. thanks much.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Hey, thank you for watching. Glad to hear that you enjoyed the video. Unfortunately I can't help you debug your solutions. One thing that comes to mind to to ensure that you clicked "add dynamic content" and put this in the expression window, not in the field itself. Otherwise it's treated as a text.
@santhoshreddykesavareddy1078
@santhoshreddykesavareddy1078 3 жыл бұрын
@@AdamMarczakYT 😊👍🏼 Thanks Adam. But really great channel to follow. Thank you
@kenpowers2767
@kenpowers2767 3 жыл бұрын
Thank you for this, this is just the approach we need for our project. I am having one small issue though. If I use "@{activity('Execute Pipeline').output}" for "message", I get this error: {"error":{"code":"InvalidRequestContent","message":"The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered: p. Path 'message', line 3, position 14.'."}}. If I use "@{activity('Execute Pipeline').error.message}" for "message", I get this error: {"error":{"code":"InvalidRequestContent","message":"The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered: S. Path 'message', line 3, position 52.'."}}. Wondering if I missed a step somewhere.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
My pleasure! You probably missed some step, like linking error path (red line) to error message. Your error states that there is no property error.message on the object you are referring to.
@kenpowers2767
@kenpowers2767 3 жыл бұрын
@@AdamMarczakYT Thanks for the quick response. Let me check that.
@kenpowers2767
@kenpowers2767 3 жыл бұрын
@@AdamMarczakYT It looks like they don't allow us to pull the error messages from the pipeline any more. Found this as I've been researching it (scroll down to the comment from douglaslMS): github.com/MicrosoftDocs/azure-docs/issues/18481 Going to try to see if I can capture output from Azure Monitor as they suggest.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
It worked in the past. I don't think they would remove this feature, in my opinion they are talking about returning error as the output of ADF pipeline, which isn't supported.
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Make sure to pass proper JSON to Web Activity like {"Message":"@activity('Execute Pipeline').error.message"}
@shivanidubey1616
@shivanidubey1616 3 жыл бұрын
Hi Adam what is the alternate of logic apps to send notifications...
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Why do you need an alternative?
@shivanidubey1616
@shivanidubey1616 3 жыл бұрын
@@AdamMarczakYT since logic apps is chargeble so mangment is asking alternate Sir
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
Everything costs money. Check out sendgrid free tier maybe. Or use Microsoft Graph REST api to send email via exchange.
@shivanidubey1616
@shivanidubey1616 3 жыл бұрын
@@AdamMarczakYT thanks Adam
@Alpa0912
@Alpa0912 3 жыл бұрын
Hi Adam - can we attached file ?
@AdamMarczakYT
@AdamMarczakYT 3 жыл бұрын
You can attach files in logic app yes. It's a little tricky to do from ADF though.
@Alpa0912
@Alpa0912 3 жыл бұрын
@@AdamMarczakYT Thank you Adam . I am planning to pass blob path from ADF to Logic apps and Then let Logic app to get content from Blob for attachment . Please let me know if you have different ideas.
@Exclusive_FiveM_Scripts
@Exclusive_FiveM_Scripts 4 жыл бұрын
Chyba zrobiłem kupę
@AdamMarczakYT
@AdamMarczakYT 4 жыл бұрын
Ciezko powiedziec czy film az tak dobry czy tak zly, ale dziekuje za ogladanie! :)
@Exclusive_FiveM_Scripts
@Exclusive_FiveM_Scripts 4 жыл бұрын
Adam Marczak - Azure for Everyone Mocny
Azure Data Factory | Copy multiple tables in Bulk with Lookup & ForEach
23:16
Adam Marczak - Azure for Everyone
Рет қаралды 188 М.
Process Excel files in Azure with Data Factory and Databricks | Tutorial
34:14
Adam Marczak - Azure for Everyone
Рет қаралды 116 М.
Алексей Щербаков разнес ВДВшников
00:47
Son ❤️ #shorts by Leisi Show
00:41
Leisi Show
Рет қаралды 11 МЛН
КТО ЛЮБИТ ГРИБЫ?? #shorts
00:24
Паша Осадчий
Рет қаралды 1,3 МЛН
23. Send Email alert when Pipeline fails in Azure Data Factory
21:59
Azure Logic Apps Control Actions Tutorial | Loops, Conditions, Parallel Runs
27:03
Adam Marczak - Azure for Everyone
Рет қаралды 53 М.
Azure Key Vault Tutorial | Secure secrets, keys and certificates easily
18:43
Adam Marczak - Azure for Everyone
Рет қаралды 177 М.
Azure Data Factory Tutorial | Introduction to ETL in Azure
24:59
Adam Marczak - Azure for Everyone
Рет қаралды 845 М.
Azure Files vs SharePoint: Best Document Storage Option?
14:58
Jonathan Edwards
Рет қаралды 24 М.
1. How to Send Email from Azure Logic Apps
17:53
WafaStudies
Рет қаралды 35 М.
Azure Data Factory Triggers Tutorial | On-demand, scheduled and event based execution
21:39
Adam Marczak - Azure for Everyone
Рет қаралды 99 М.
How to Run Databricks Notebooks using AzureDataFactory
20:56
GeekCoders
Рет қаралды 6 М.