Speech AI on Jetson Tutorial

Full article on JetsonHacks.com: wp.me/p7ZgI9-3LZ
This is an involved installation process, hence the video. Beginners might find this difficult.
RIVA Quick Start Guide: docs.nvidia.co...
NGC: ngc.nvidia.com
NGC CLI Tool: ngc.nvidia.com...
RIVA Python Clients: github.com/nvi...
Videos:
Sign up for NGC and Generate API Key: • Getting Started with N...
ngc config set walkthrough : • Getting Started with N...
RIVA is a SDK for developing conversational AI Speech. RIVA can run on-device without the need for an Internet connection. There are a couple of undocumented pitfalls in the installation process which we cover here. The full set of commands that we executed are in the JetsonHacks article.
RIVA is a comprehensive library with includes:
• Automated speech recognition (ASR)
• Text-to-Speech synthesis (TTS)
• Neural Machine Translation (NMT) (language to language translation)
• A collection of natural language processing (NLP) services, such as named entity recognition (NER), punctuation, and intent classification.
Here we install the RIVA SDK on a NVIDIA Jetson Orin Nano Developer Kit. If you are using this device, make sure that the QSPI memory has been updated properly. The available memory should be ~ 7.2 GB. This install can be a little challenging on this 8GB device, as RIVA and models take ~ 6.8GB.
RIVA is in beta release on the Jetsons, and the instructions here may need to be changed to work in future releases. RIVA works on the NVIDIA AGX Orin, Orin NX and Orin Nano. RIVA also works on the NVIDIA AGX Xavier and Xavier NX. You must have 8GB+ to follow along with the demo in the video. RIVA requires JetPack 5 or above.
Equipment in video:
NVIDIA Jetson Orin Nano Developer Kit: amzn.to/3YnMl8Q
Logitech Headset with microphone: amzn.to/3OWK1CN
The Laws of Human Nanture - Robert Greene : amzn.to/3YsS7pJ
Kamala Harris Motivational Poster: thefederalist....
Amazon JetsonHacks Storefront: www.amazon.com...
Website: jetsonhacks.com
Sign up for the newsletter! newsletter.jet...
Github: github.com/jet...
Github: github.com/jet...
X: / jetsonhacks
Some of these links here are affiliate links. As an Amazon Associate I earn from qualifying purchases at no extra cost to you.

[@ArztvomDienst]

THIS is how I want Tutorials to be. No drama, no BS, just boiled down necessary information (and a grain of humor). Instant Sub.

[@JetsonHacks]

Thank you for the kind words. To be fair, some of the videos only have a sub grain of humor, depending on the allocation for the week. Thanks for watching and subscribing!

[@InsanityisSanity]

Just ordered an Orin AGX. Cant wait to start learning

[@JetsonHacks]

Congratulations! Lots of exciting work ahead. Thanks for watching!

[@harry1622]

First of all thank you for the great video, you helped through my first ai deployment I had a laugh seeing mine recognizing your voice testing yours through the speaker

[@JetsonHacks]

You are welcome, and thank you for the kind words. That's a great story! Thanks for watching!

Pretty straight forward. Thank-you Jim.

[@JetsonHacks]

You are welcome. This took me a couple of months to get it to reliably work, I think it is worth sharing with folks. Thanks for watching!

[@timothy6537]

Thanks you!

[@JetsonHacks]

You are welcome, and thanks for watching!

[@HSHighScore]

woah, ty to tutorial :D

[@JetsonHacks]

You are welcome, and thanks for watching!

[@BenjiNotknown]

How could somebody use this method to take the transcribed text then somehow auto translate it into another language? For example, automatically transcribe Russian into English? Is it possible using the available example scripts?

[@JetsonHacks]

The RIVA SDK has a component called Neural Machine Translation (NMT). For example there's English to Russian and Russian to English models to use. Here's a link to the documentation: docs.nvidia.com/deeplearning/riva/user-guide/docs/tutorials-nmt-contents.html This is a professional level SDK. 'Example Scripts' may be quite involved depending on what you are trying to do. There is a large amount of sample code. Thanks for watching!

[@eranfeit]

Hi, Just checking . Would it run on the Jetson Nano as well ? Eran

[@JetsonHacks]

The RIVA SDK is for JetPack 5 and up. Unfortunately it doesn't run on the Jetson Nano. Thanks for watching!

[@JetsonHacks]

You may want to check out Nemo: github.com/dusty-nv/jetson-containers/tree/master/packages/nemo

[@stevephillips5392]

Hi, another great video! Do you think I would encounter any problems trying this on a Jetson Orin Nano Devkit running Jetpack 5.1.2?

[@JetsonHacks]

Thank you for the kind words. Not any issues that I know of, but I haven't tried it yet. Things are always a bit iffy when there are version changes. Thanks for watching!

[@stevephillips5392]

@@JetsonHacks I did try this on the Jetson Orin Nano earlier with Jetpack 5.1 and failed, but tried it again with Jetpack 6.0 and now everything is working flawlessly. I don't see RIVA support being listed for Jetpack 6.0 but it works. Now to add in a Small Language Model like Sheared-LLaMA-2.7B-ShareGPT.... Thanks again for this excellent tutorial!

[@JetsonHacks]

@@stevephillips5392 You are welcome. I'm glad you got it to work. Sounds like a fun project!

[@Maxwellpaulwall]

Couple questions: 1. How much does this cost if you deploy it on your own Jetsin Orin Nano hardware? Is the model free if you deploy it yourself. (I know they charge you if you're using the A100s in the cloud, but what about this?) 2. When riva is loaded and running, can you seemlessly to speech to text and text to speech without major delays? Or does each module need to be loaded individually (overloading the capacity of this board to run both at the same time, or hugely delay any application relying on both text-to-speech and speech-to-text to occur simultaneously in real time)

[@JetsonHacks]

I do not know the answer to your questions. My understanding is that you must pay for RIVA is you meet a certain threshold of use. Please contact NVIDIA for more information. RIVA is in beta on the Jetsons, it's difficult to tell how it will perform in a release form. Thanks for watching!

[@irem2719]

@@JetsonHacks is there a fee for using RIVA with Orin? Like if i have nvidia jetson orin do i have to pay for riva ?

[@JetsonHacks]

@@irem2719 I do not know the answer to your questions. My understanding is that you must pay for RIVA is you meet a certain threshold of use. Please contact NVIDIA for more information. RIVA is in beta on the Jetsons, it's difficult to tell how it will perform in a release form. Thanks for watching!

[@Luiblonc]

Will this work on the Jetson nano? I am interested on running this on a Jetson nano that I already have, and was wording if I could configure it to work as speech to speech recognition.

[@JetsonHacks]

This particular application requires a Jetson Xavier or Jetson Orin to run due to the processing and memory requirements. Thanks for watching!

[@YigalBZ]

Great video, thank you. Can Jetson Nano implement the Speech AI and do real time image detection same time? The memory consumption may be a problem....

[@JetsonHacks]

The RIVA SDK does not run on the Jetson Nano, it runs on the Jetson Orin Nano. There are other alternative speech packages for the Jetson Nano. It's hard to tell about a specific application, but lack of memory is going to be an issue on the Jetson Nano. Thanks for watching!

[@YigalBZ]

@@JetsonHacksThe video was refering to Jetsons, so was the article without a reference to the Jetson Nano - perhaps that should have been mentioned.

[@JetsonHacks]

@@YigalBZ Thank you for the suggestion. This is in the video at the 0:40 mark.

[@SmnHashemi]

Hi. Thank you for the video. Although I am following all the steps as shown in the video, I am getting “Health ready check failed” Error after running the “bash riva_start.sh” command. I tried with both versions of RIVA 2.12 and RIVA 2.14 I have NVIDIA Jetson AGX Orin board and nvidia-jetpack version 6.0-b52 Do you have any idea how should I resolve this?

[@JetsonHacks]

I do not believe that RIVA supports JetPack 6 yet.

[@EstebanGallardo]

@@JetsonHacks Riva 2.15.0 is now supporting JetPack 6. I tried the tutorial but with Riva 2.15.0. but I get the same error: "Health ready check failed". Are we missing something new? I can share any information you need from my side to solve this issue.

[@QiicKellen]

@@EstebanGallardo Hi.Have you solved it? I have the same problem.

[@seawolfmx]

Can you make a video in windows 11? Can you translate tris video to spanish?

[@JetsonHacks]

I do not know what you are asking. I do not know what 'video in windows 11' means. Unfortunately I don't know Spanish, so you will need to rely on other translation tools.

[@yunusozd]

is it time to upgrade my TX2?

[@JetsonHacks]

Could be! The Orins are definitely the go forward path for the Jetsons. Thanks for watching!