36. Continuous Integration and Continuous Deployment in Data bricks

[@murtazasomani367]

very nicely explained - can you please share YAML code to create the pipeline?

[@eric-seastrand]

Seriously the best explainer about how to *actually* do automated deployment with Databricks. Better than their first party stuff. They should just hire you..

[@muralikrishnarayudu3095]

You explained very nicely the point of point. Thanks for sharing this topic. It really helped me to grab the knowledge in this area

[@ayushvarma9657]

Terrific explaination! I would watch it again to grasp it even better🧑‍🎓

[@neelbanerjee7875]

Thanks for all these valuable contents. Requesting you to make in depth practical video on the below topics - 1. Unit testing of databricks notebook (pyspark) 2. Real time Issue faced in production environment by pyspark developers

[@sravanreddyboina6752]

Excellent video, thank you! Can you please make a video on how to handle CI/CD when there are two or more environments and with unit testing if possible.

[@IndianMusaafir]

Good Info. Topics covered in detail. Able to get the information that I was looking for. Thanks

[@SakshiAgrawal-c9s]

Amazing explanation, can you please provide the pipeline code or any link where I can find this code.

[@tarunacharya1337]

Very nicely explained - I was going in circles reading the documentation, your presentation saved me a tonne of time. You have great presentation skills, keep up the good work :) Can you please share the YAML code to create the pipeline?

[@aswinibarik244]

Tarun Acharya , Did you ever get the YAML Code ?

[@tarunacharya1337]

@@aswinibarik244 yes - currently parses ok but I need to install Databricks CLI and a particular command in the script fails as the CLI is not installed.

[@tarunacharya1337]

@@aswinibarik244 currently got this working with a devops task for data bricks when i create my pipeline using the UI( skipped yamal)

[@akshay11000]

@Tarun Acharya - is your yaml working fine?

[@Soumadip.B]

You have shown only the CI part which is Build pipeline. Where is the CD part i.e. release pipeline?

[@vaibhavkarade-mg6mi]

Great explanation. Can you share the YAML code for reference.

[@somesh512]

A fantastic video I must say. I want to know One thing. Now for deployment in test to prod, do we just change the yaml script where source is test and target is prod

[@satishgs5355]

Could you please make a video how to deploy code using Jenkins CICD

[@narendrakishore8526]

Great explanation. This might be useful for anyone who are learning. Is there any udemy course to subscribe for detailed CICD related

[@yasoram8007]

Hi Bhawna, Thank you for amazing information in clear format , Is there any Azure Data factory tutorial series

[@soucianceeqdamrashti8175]

Hello, awesome video! Good overview of how to use Repos in databricks! Is your YAML file available anywhere? Thanks.

[@NripaEmpowerthroughknowledge]

very insightful content

[@jyotirmoyroy8619]

Can the YAML code be shared

[@NaveenKumar-xs9ro]

Thanks for the valuable content.

[@veerendra_1996]

Great Explanation, Thank You!

[@ramanthraj]

Thanks for all these contents. Requesting you that make video or share me any script for databricks jobs start stop using azure devops

[@skasifali4457]

It seems that when we try to push modifications to a file that has already been uploaded in DevOps, it is causing issues, resulting in the creation of a new file with the name '1'.

[@syedsikindar4539]

Nice explanation

[@lakshminarayananr3799]

very nice. How to do CI/CD release for only modified or newly created files instead of deleting and creating all in notebook release folder.

[@MonikaTyagi-f2s]

Amazing content, solved most of my doubts , Can you pls share the YAML script as well ?

[@karanarora13]

Yes , we need Yaml file if possible share on video description

[@biplovejaisi6516]

Also how can we point to a specific branch in azure DevOps that contains a notebook that needs to be triggered by adf?

[@pradeepp9126]

A very detailed explanation 👌 😊 especially on the yaml file creation part great job

[@thinkbeyond18]

you didnt do a demo to push the code from new branch to main branch in the YAML

[@abdelazizfeddaoui1993]

Big Thanks, very nicely explained!!!!

[@anujgupta-lc1md]

you missed approval part, In previous way we create a separate release pipline with UI & approval process.. This new way will directly push my code to QA and prod if any changes made to repos which creates a problem when you work in Agile manner ideally all developer should push code to CI and then on a particular day all code will move to QA by a separate release pipeline with approval. May be you can comment more not 100% by this new way how did you manage Agile and approval.

[@jimzhang2562]

Thank you so much. Really helpful.

[@anantababa]

Very nice explanation, are you providing any live training for databricks

[@satya0502]

thanks for this wonderful explanation

[@imranhossain1660]

Easy to follow tutorial. Can you please share the YAML file? It will certainly help many of us.

[@IndianMusaafir]

Can you please share the yaml file for reference. We are getting stuck while creating the yaml scipt. Thanks

[@santoshk4530]

Superb training

[@ravitejmadem]

Is there a way to extract only newly added files from git and deploy to databricks 1. lets say we have 2 files newly added/modified from existing notebooks and committe dto main 2. CI pipeline has to cherry pick only those 2 files and process it and then deploy back to databricks Note: there should not be any deletion in the existing release folder in databricks , but just add newly modified or added files. Can you please help me with the code/ process

[@prashantbharad4499]

its very informative Bhawana ma'am !!! liked and subscribed 👍 Thanks.🙏

[@shivakumartalwar79]

For clip 35:57 , for your demo , Databricks able to configure but same approach am getting aborted ! With error return code : 1 . , please help

[@lokeshloky1048]

Thank you so much yar .. it helped me a lot and subscribed ❤

[@shishirkumar9420]

very nice explanation !! thank you very much !!

[@yasharthsrivastava7562]

Very informative and relevant too, was waiting for this topic :))

[@sravankumar1767]

Nice information u have shared 👍 😀

[@manjit_singhh]

Very informative. Will you be showing the CD part as well.?

[@vkincanada5781]

@Cloud Fitness - Demonstrated right deploying code to DataBricks. I believe that is CD!

[@neelbanerjee7875]

@@vkincanada5781 yes.. but it would be good if Demonstration can be given to deploy in multiple environment like test/prod.. that would clear out the whole concept including how to handle with different environment variable.. @Cloud Fitness - if possible pls extend this video to the above point..

[@sreelakshmisandeep90]

Could you please share the YAML file ?

[@samuelsamm9297]

Hi , this very great video. Thanks a lot for detailed explanation. can you please send us the YAML file.

[@neelbanerjee7875]

Could please share the yaml?

[@Sainik-Vihar-Layout-Yelankha]

what you explained its only CI , but where is CD pipeline ,

[@durgaprasadv-tc6lq]

Thanks for the video , Can you pls share the YAML script as well ?

[@sarveshkr5082]

thanks please YAML files?

[@lucaslira5]

Thank you so much for this video. Can you shared the yaml file? Please

[@leonardomauricio3149]

Nice. Thanks!😃

[@udayshankarlakkakula]

Can you please share YAML Code

[@avaneeth36]

Yml file please?

[@akshay11000]

Could you please share the yaml file

[@ajinkyabhabad1465]

Great tutorial! Can someone share the YAML file used in the tutorial? Thanks!

[@amannandgaule3572]

Pls share yaml file

[@chetan9741]

Has anybody here have this yaml file. kindly share, TIA!