Setting Up a RAG Demo on NVIDIA AI Workbench

[@JMai-ci9nl]

This is so far the most complicated RAG system setup I have ever seen. Good luck to anyone who is going to actually set it up and use it.

[@raiskazi9760]

Its one time setup. I found this video tutorial easy to follow and it worked for me without any hassles.

[@raghums1355]

Nice video, could you maybe explain the code as to how the Apis are used to interact with NGC?.

[@Root18D]

Host Mount Configuration Required In order to proceed, you must configure the source directories below. Target Directory /project/models/ Source Directory *

[@carlosrm8091]

Excuse me Sir. I have get the Application Error you describe. Im unsure If I have correct listened... You said I shall update BIOS? Isnt it? I dont see any relationship between BIOS and WSL2, as I have run Ubuntu over W11 in this PC before... Is it needed to install WSL nvidia graphic drivers to been able to use CUDA in workbench?

[@NVIDIADeveloper]

Hi there. If you get an error installing WSL2 you could try first to install it manually using wsl --install. If this works, then you can just run the workbench install again and it should recognise that WSL2 is already installed. If a manual install doesn't work, it's often because Virtualization extentions aren't turned on in your PC BIOS. The name of the setting varies depending on which BIOS you have but look around the CPU settings. I also hear that there may be some conflicts between Hyper-V and WSL2 so try to disable Hyper-V. If you're running Windows itself inside a VM, you might have other problems with WSL2 and nested virtualization. As this is a WSL2 error, I'd search on the web for articles about WSL2 and virtualization. It isn't anything to do with the NVIDIA drivers in the Linux VM. This demo doesn't need those because the inferencing is done by NGC in the cloud, not locally. Thanks for watching and good luck!

[@carlosrm8091]

Disabling hyper-v didn't solve the issue. WSL2 is installed, and i can check it over powershell or cmd. Nonetheless,when double clic the nvidia desktop link to AI workbench, it says "Application Error" "something happened, please contact support or try restarting the application And installer. The PC is AMD Ryzen 7 5800X 4.2 GHz, 32GB Windows 11 Pro 23H2, RTX 3060Ti. I ve tried on a different PC at the University and I get the same issue

[@tylerwhitehouseus6310]

@@carlosrm8091 hard to say what the issue is without looking at the application logs. KZbin won't let me post a link to the Issues forum, but if you search "NVIDIA Forums AI Workbench" you will see some posts that may help.

[@yunchaoyang8464]

The open chat cannot launch the webui. the output for chat saying that: stat: cannot statx '/var/host-run/docker.sock': No such file or directory groupadd: invalid group ID 'docker' usermod: group 'docker' does not exist Any idea how to fix this?

[@NVIDIADeveloper]

Hi! Let me speak to somebody and get back to you to see if it's a known issue.

[@tylerwhitehouseus6310]

The Hybrid RAG Project requires Docker as the runtime. It won't work with podman. Which runtime did you select when you installed?

[@yunchaoyang8464]

@@tylerwhitehouseus6310 Thanks. I selected podman. I think this is the reason for the error I saw.