MLOps Tutorial #4: GitHub Actions with your own GPUs

Tools like GitHub Actions and GitLab CI automate repetitive aspects of software development- and they can also automate machine learning tasks like model training, testing, and reporting . By default, these tools provide CPUs for running workflows. This tutorial will show you how to set up a GPU (on-premise or cloud) as a self-hosted runner using the CML Docker container, which comes ready with CUDA drivers and software to run GitHub Actions and GitLab CI workflows.
This tutorial is designed to be beginner-friendly, but we recommend watching the previous videos in this series for more ideas and project inspirations about using continuous integration in ML.
*Helpful links*
Blog about using the CML docker image to setup a self-hosted runner with GPU support: dvc.org/blog/c...
GitHub Actions self-hosted runners docs: docs.github.co...
Discord channel for technical support and questions: / discord
🧑🏽‍💻 To learn more. about our tools, take our free online course at learn.iterativ...

[@dvcorg8370]

Please note we have deprecated the dvcorg/cml-py3 container image. You can get the same results with: - container: docker://dvcorg/cml-py3:latest + steps: + - uses: actions/checkout@v3 + - uses: iterative/setup-tools@v1

[@chenjus]

Woot! Love these videos. Can't wait for the next one!

[@jackbauer322]

More interesting and entertaining than Netflix ! Thanks again !

[@jamesmowatt3115]

Excellent tutorial, is it possible to use this with Organisation level runners? I can see that CML supports tags, is it possible to setup the runners at an organisation level and access them from the project? I.e. not specify the project URL when running the docker container

[@davidgortega3734]

Hi James! Not actually, runners subscribe to a particular repo. It could be a very nice feature to have would you mind to open a ticket?

[@visenger1]

Thanks for your awesome work and this video!

[@sureindia]

Hello Elle and Deevee, do we have a video for step-by-step enabling and connecting AWS EC2 based machine for a cloud case self-hosted runner? I am a little behind on configs with AWS and would like to go through the pre-steps and follow this video. Appreciate this MLOPS playlist, awesome.

[@purushothamangempuraj177]

Great videos.. I'm glad, i found DVC and CML. They are solving key problems with my ML workflows! Thanks for the community.

[@efexzium]

Love your work! Elle and Deevee

[@mprone]

so basically there's no way to require a *remote* runner to have and use a GPU ?

[@dvcorg8370]

If by remote runners you mean those that are provided by GitHub and GitLab, as opposed to self-hosted; yes, none of them provide GPU accelerators. That's the problem we solve with cml-runner as long as you have a GPU on your computer or a cloud account on AWS, Azure or GCP. 👍🏼. Join our community in Discord to get all your DVC and CML questions answered! 👉🏽 discord.gg/rpgRdvfyAf

[@jackbauer322]

I still feel that a UI like MLFlow, Neptune and so on would be very helpful and appreciated to make pipelines visually ! :)

[@dvcorg8370]

Yeah, it's a good idea! Visualizing the DAG can really help with pipelines. Passing on this request to the team :)

[@jackbauer322]

@@dvcorg8370 And also a bit of interactivity to create it through the UI and modify parameters ;) Thank you so much for your kindness , smile and openess ! :)

[@RedShipsofSpainAgain]

+1 for this feature. Visualizing the pipeline via a UI would be of definite value. Thanks Elle!

[@sandeep-mistry]

Awesome tutorial! Will you be covering how to use AWS SageMaker training jobs instead of ec2 instances in future videos?

[@dvcorg8370]

Thanks Sandeep! We don't have anything lined up yet, but we'll investigate SageMaker ;)

[@dataengineeringmelbourne5897]

@@dvcorg8370 +1 for this request. SageMaker or EMR anything would do.

[@sreeragm8366]

@DVCorg Asking for my friend: Is it possible to use Colab GPU as a runner?

[@dvcorg8370]

We've never seen anyone do it, but they could try! You can add a machine as a self-hosted runner using the method here: dvc.org/blog/cml-self-hosted-runners-on-demand-with-gpus I'm imagining that as long as they can install dependencies (like nvidia-docker) and run these Docker commands from the Jupyter notebook (usually done with a `!` in front), it could work. The downside is they'll have to "spin up" the notebook and re-connect to GitHub Actions (runners have to be "listening" for jobs) manually when they want to use it. But could be worth a shot for some situations- let us know how it works?

[@khursani8]

​@@dvcorg8370 I think it will not work since Colab cannot install Docker(or more like can install Docker but cannot start the service)

[@1etcetera1]

commented and subscribed

[@TheBenyos]

Excellent tutorial, thank you! Any chance you have some useful ref for working with self-hosted runners in an auto-scaling environment so that one doesn't have to manually start/stop the runner?

[@dvcorg8370]

Yes, we have a use case with Docker-Machine to autodeploy self-hosted runners. They turn themselves off when the job is done. github.com/iterative/cml_cloud_case If this approach wouldn't work for you, we'd love to know more about your setup- feel free to open a ticket on the CML repo and make a feature request.

[@davidgortega3734]

Hi Matan, as @DVCorg says we have an example on how to setup a provisioner job with docker machine previous to the train. Which kind of auto-scaling environment do you expect? Its a vendor like AWS or a private cloud?

[@TheBenyos]

@@davidgortega3734 Thanks! I'll take a look at the Docker-Machine solution. I was thinking about using my own self-hosted GitHub Actions runners in AWS so I will be able to run my pipeline using GPU acceleration and also save some costs on the way :-)

[@carloslopez7204]

In which part of the workflow do you do Data Exploratory analysis. Do you use notebooks for that? How do you track them?

[@dvcorg8370]

For exploratory data analysis, when you really want a lot of interactivity and quick exploration, you might want a notebook! Or some fast scripts on your local machine. Whatever you're most comfortable with. If you're doing notebooks and want to track them with Git, checkout the `nbdev` library. CI/CD is a great way to "check" your data/models into Git and test them in a standardized environment whenever you're ready. For some people, that can be after manual investigation on their local machine.

[@doodah120]

Hi, thanks for the awesome video! Why do you need the docker? Following the instructions on github, I've managed to create my self hosted runner without docker, but my action crashes with : "cml-send-comment: command not found"

[@dvcorg8370]

Good question- our Docker has the CML library installed, so you can run `cml-send-comment` and similar functions. But you don't *need* Docker- you can also get CML with npm: github.com/iterative/cml#install-cml-as-a-package once you do the npm install, you should be able to run CML functions without hitting that error.

[@doodah120]

@@dvcorg8370 oh very cool, thanks a lot!

[@woma9631]

Hey, I really like your tutorials. Is it possible to use all this with singularity containers, as some HPC clusters do not allow docker. This would be incredibly helpful. Thanks in advance for your answer.

[@dvcorg8370]

Although we've never tried with singularity containers, you can install CML with npm and bypass using Docker!. Docs are here: github.com/iterative/cml#install-cml-as-a-package

[@woma9631]

@@dvcorg8370 Nice, thank you - I think your docker container could also be converted via github.com/singularityhub/docker2singularity - Which brings me to another question - Is the continuous integration with git still possible when using singularity - so a runner in a singularity container, where no docker is installed on the system? - according to gitlab.com/gitlab-org/gitlab/-/issues/16269 this is still an open issue - This could be interesting, souchal.pages.in2p3.fr/hugo-perso/2019/09/20/tutorial-singularity-and-docker/ , also I am not sure if it solves the problem

[@dvcorg8370]

@@woma9631 If I understand correctly, yes, you could have a singularity container with docker running CML. You'd just want to do `npm i -g @dvcorg/cml` before running your workflow to get the library! Docker doesn't need to be installed on the system. I think you might be our first singularity user, so if you try this.... keep me posted how it goes! -Elle

[@raghavaalajangi2136]

Hey, any updates regarding the singularity container approach?

[@NS-gr9cy]

What a fun person to eat fries with..

[@jalaj1]

Hi can you make video on mlcertific.com It is providing free certification on MLOps

[@neurondeep]

I love u