Use These! Jetson Docker Containers Tutorial

NVIDIA Jetson Containers are one of the most valuable tools for developing on Jetson, from Jetson Nano to the Jetson Orins. Full article on JetsonHacks: wp.me/p7ZgI9-3MT
Docker is the container manager on Jetson. This allows you to package together your application along with libraries, system and environment settings together. This Image can then be run sandboxed independent of the rest of the system.
In this tutorial, we introduce some of the important concepts that will help you get comfortable with using containers in your daily use.
Shown in the video:
NVIDIA Jetson AGX Orin Development Kit: amzn.to/3Er8Roe
Monty Python and the Holy Grail: amzn.to/3P59A31
00:48 Introduction to Docker
01:43 Why use Jetson Containers?
02:08 l4t-ml container. Jupiter Notebook with PyTorch amongst others
02:35 Jetson Zoo
03:50 System Setup
05:25 Stable Diffusion
07:14 Llama GPT
As an Amazon Associate I earn from qualifying purchases.
Visit the JetsonHacks storefront on Amazon: www.amazon.com...
Visit the website at jetsonhacks.com
Sign up for the newsletter! newsletter.jet...
Github accounts: github.com/jet...
github.com/jet...
Twitter: / jetsonhacks
Some of these links here are affiliate links. As an Amazon Associate I earn from qualifying purchases at no extra cost to you.

[@JetsonHacks]

Oops! At 4:47 this should read 'sudo gedit /etc/docker/daemon.json' The autocorrect on the Mac doesn't like Linux! Bad Siri.

[@samrubenabraham6979]

I must admit, this tutorial is pretty much what I needed! Thanks a lot!

[@JetsonHacks]

You are welcome, and thanks for watching!

[@DienNguyen-pm6fi]

the intro made me think i open the wrong video lol. Thank you for your detail explanation

[@JetsonHacks]

Always helps to keep people guessing. You are welcome, and thanks for watching!

[@SuperMakeSomething]

Fantastic video as always, Jim -- a big thank you to you and Dustin! I needed to get ROS up and running on my Jetson Orin Nano and have been wanting to explore Docker for a while. Your instructions and the pre-made containers made the process super easy! Docker is great!

[@JetsonHacks]

You're too kind. Dusty does some pretty incredible work in a fast changing field. I'm glad you were able to get up and running quickly! I hope you're making videos on your new project soon.

[@ardavaneghtedari]

Like all your previous videos, this is very useful. Thank you.

[@JetsonHacks]

Thank you very much for the kind words, and thanks for watching!

[@richardleon5149]

Good job! Thank you for sharing 😊

[@JetsonHacks]

You are welcome, and thanks for watching!

[@anthonylobay9345]

Thanks!

[@JetsonHacks]

Thank you for your donation! It's good to know that you find this useful. Also, thanks for watching!

[@ardavaneghtedari]

Thanks!

[@JetsonHacks]

Wow, thank you very much for your generous donation! I'm glad you found this useful.

[@patis.IA-AI]

Thanks Jim , very usefull and clearly explained

[@JetsonHacks]

You are welcome, and thanks for watching!

[@sansmojo]

Shout out to dustynv. I've been using his containers on jetsons for years at this point.

[@JetsonHacks]

Double thumbs up on that. Dusty is the best! Thanks for watching!

[@louieearle]

This is fascinating. Gives me a new appreciation for the token speed I get from Chat-GPT 4.0. I will definitely be taking advantage of these docker containers. The whole Nvidia/Jetson ecosystem is a lot to wrap ones head around.

[@JetsonHacks]

You can see why the bill is so high to serve LLMs, you have to have a lot of compute power. There's so much money and effort being poured into training, inferencing isn't keeping up. The Jetson ecosystem has been changing so fast over the last few years that it's difficult to have an overview. With the introduction of different generative AI models what seems like weekly, about the only way to get started is to just jump in on a subject you like and dig in. Thanks for watching!

[@yawarrehman7139]

great content

[@JetsonHacks]

Thank you for the kind words, and thanks for watching!

[@HSHighScore]

woah ty :)

[@JetsonHacks]

You are welcome, and thanks for watching!

[@KILL3RYT]

Hi , I use jetson Orin nano 8gb, can you tell me how to stop usb auto suspend please

[@JetsonHacks]

Depends on what you're trying to do. If you only want to allow one particular device to autosuspend, you can add a udev rule in /etc/udev/rules.d/ If you want to autosuspend usb entirely, you would modify /boot/extlinux/extlinux.conf : * Go to the “/boot/extlinux/extlinux.conf” file. * Look for the “APPEND” key/value entry. * At the end of that entry add a space and then “usbcore.autosuspend=-1”. (No quotes) * Reboot, verify the “usbcore.autosuspend=-1” shows up in “cat /proc/cmdline”. * Test. You can also modify the /etc/rc.local file, or add flags to the appropriate /sys/bus/usb/devices/.../power/control You can find these answers in the official NVIDIA Jetson forums. Thanks for watching!

[@asagrote998]

Awesome video. Quick question: Why are you using a virtual environment? What does it mean and what are the benefits?

[@JetsonHacks]

A Python virtual environment is a self-contained directory that contains a Python installation for a particular version of Python, along with additional packages. This environment is separate from other Python environments on the same machine. Python virtual environments are a key tool for effective Python development, allowing for dependency management, project isolation, consistent development environments, simplified deployment, ease of management, and operation without root permissions. Thanks for watching!

[@bheemkill4282]

Hy jim, can you show how to save jetson Orin nano's os and other applications installed in one board into an image file so that i need not install all libraries each time and i can use the image file to write into another jetson Orin nano? Please...

[@JetsonHacks]

You should ask this question on the official Orin Nano developers forum. AFAIK, you need to add the libraries you need into a stock image and build it. Good luck on your project!

[@JoséMiguelRegoTerol-y9v]

Greetings! I have seen a warning when running the LLM Docker container: You are potentially exposing the web UI to the entire internet without any access password. You can create one with the "--gradio-auth" flag like this: --gradio-auth username:password Make sure to replace username:password with your own. Do you know where can I add this flag? Thanks a lot!

[@JetsonHacks]

Did you try adding those commands to the command line when you launch the Docker container?

[@ngocmanprocoder]

Hello, how can I connect my current project using Docker? Thanks a million.

[@JetsonHacks]

Use the -v option to mount your project directory into the Docker container. Replace /path/to/your/project with the actual path to your project directory. If you're using Jetson containers: $ jetson-containers run $(autotag l4t-pytorch) -v /path/to/your/project/:/workspace Once inside the Docker container, navigate to the mounted directory to access your project files.For example $ cd /workspace

[@ngocmanprocoder]

@@JetsonHacksversion of container image will be automatically selected to suit my existing project as using autotag, right? Thanks.

[@JetsonHacks]

@@ngocmanprocoder It will attempt to match the image with the version that matches your version of JetPack.

[@ngocmanprocoder]

​@@JetsonHacksfinal question is i want to create docker container with opencv (with CUDA support), tensorflow. How do i type the command? Thanks.

[@JetsonHacks]

@@ngocmanprocoder What have you tried? From your description, it sounds like the l4t-tensorflow package fits. If you need help, please ask your questions in the official NVIDIA Jetson forums where people can better help you.

[@TasosManolidis]

Amazing video i am really newbie to all of this.but how can I use ros 2 container and made my own environment so I can set all my nodes there?thanks again for all your videos😊

[@JetsonHacks]

You can develop inside the container and save it as a new image. Or you can write an extended Dockerfile. It depends on which development methodology you are most comformtable with. Some people use VS Code on the Jetson, other run the Jetson headless and use code editors on remote machines. It's a pretty big subject.

[@ngocmanprocoder]

Hi, the process of pulling container image depends on the version of jetpack we are using. If this is the case, my attempt to execute any application that requires Tensorflow 2.5.0-which is incompatible with Jetpack 5.1.2-like Facial Recognition using Facenet would fail. What should I do if this happens? Many thanks.

[@JetsonHacks]

I do not know how to answer this question. If you have specific questions about this topic, please ask in the official NVIDIA Jetson forums.

[@yaoyuan5176]

Hi, how do we create our own docker images or make modification to existing docker images? Thanks a lot

[@JetsonHacks]

You begin with a Docker file, either one you create yourself or an existing one. There are Docker files in the jetson-containers repository as examples. You then do 'docker build' to create an image. Finally, you 'docker run' the image. Did you have a particular Docker image you want to make?

[@yaoyuan5176]

@@JetsonHacks Thanks for getting back! I was wondering how to utilize docker to simplify the prototyping experience. Say I want to build a small robot on ROS, I can start with the ROS docker image to have the ROS infra set up. But how can I do the prototyping development within this image and once done, replicate the image to run on another robot?

[@JetsonHacks]

@@yaoyuan5176 There are many ways to do this, depending on your development style. Some people use VS Code with the Docker extension, have an external directory they mount so that they can keep their code separate from the ROS Image. You can save a container as an Image after you're done developing. Other people will develop in the outside directory, and when they're done create a new Docker file which extends the ROS Docker container by adding their code. Then they build the Dockerfile into an Image. There's lots of tips and tricks, KZbin comments isn't a space to share them all. Good luck on your project!

[@QiicKellen]

Hello Jim , could you tell me how to build the NanoDB demo when I edit the webUI source code please ? I used command "./build.sh nanodb" but isn't change the UI.

[@JetsonHacks]

This is an area where you would be better served to ask for help in the jetson-containers community. You can ask on the Jetson forums, where are monitored by the community members, or through the jetson-ai-labs or jetson-containers website and repository. Good luck on your project!

[@rockatang2189]

Hello there. I met an error when I got in jetson-containers and tried to do the "pipe install -r requirements.txt". It says "command "python setup.py egg_info failed with error code 1 in /tmp/pip-build-tlq0u7hj/pyymal/" How can i fix this so that I can normally install the docker? Thank you.

[@JetsonHacks]

I'm assuming that you mean pip3 instead of pipe for your command. You can first try upgrading pip. You can get a better answer from the jetson-containers issue list or the official NVIDIA Jetson forums. Thanks for watching!

[@Royaltea_Citizen]

Hey Jim, is there a quick and dirty way to use these to run tools like Ollama/ollama-webui or Fooocus on GPU, Im hitting the wall where its using the CPU, or am I over simplifying it and its actually a lot more complex? Thanks

[@JetsonHacks]

The Docker images should use the GPU automagically. You can file an issue report in the Github repo if you are encountering a problem. Thanks for watching!

[@Royaltea_Citizen]

@@JetsonHacks thanks for the reply, the images provided by dusty are great and work with GPU, just want to know if there’s a way I can implement other open source projects that work with cuda but for the jetson specifically, like these projects work fine with the Nvidia desktop gpu but I’m not seeing how to change them over in docker for instance to find the jetson gpu.

[@JetsonHacks]

@@Royaltea_Citizen I don't think there's a simple, one size fits all answer here. Most of these programs are pretty complicated, and have a pretty heavy set of dependencies. Many are directed towards a PC architecture and driven towards the very most recent version of libraries. Here's an answer for something like Ollama: forums.developer.nvidia.com/t/ollama-and-jetson-issue/278708/9

[@shakhizatnurgaliyev9355]

👍👏

[@JetsonHacks]

Thanks for watching!

[@g.s.3389]

can you make a video on deepstream, pls.

[@JetsonHacks]

Thank you for the suggestion, and thanks for watching!

[@marcosramirez8056]

Somehow my jetson orin nano keep freezing on every sample a i try on it. it is getting frustrating.

[@JetsonHacks]

It's hard to tell from your comment which samples you are running. For a Orin Nano, you have to use small models.

[@FeCardoza]

Hello! I don't know if you have tried the torch-tensorrt image. Once I was inside the containr at root@[user]:# I was able to run python --version and see that python 2.7 was installed, however, there were many other dependencies that were supposed to come with the container, and when I go into python and run import torch, no such module is found. Moreover, attempting to install using pip install torch and I get an error saying pip can't be found. I find this very odd but everything I can find leads me to believe I pulled and ran the docker correctly. Particularly that 12GB was taken up on my machine. Would you mind trying the torch-tensorrt image from dusty-nv (L4T35.3.1) and letting me know if it's actually missing the dependencies or it's just me? Thank you

[@JetsonHacks]

I'm not quite sure what you're asking. The torch module is in Python3. Run python3 at the command prompt. Thanks for watching!

[@FeCardoza]

@@JetsonHacks Thanks for replying even though I made an unclear or unreasonable request. Of course it's as simple as using python 3 instead. Is there a way to contact you for paid consulting services?

[@JetsonHacks]

@@FeCardoza I don't think it was unreasonable. I can only work for companies, not individuals.

[@FeCardoza]

@@JetsonHacks I run my own, albeit small, company. How do I contact you?