NVIDIA ChatRTX: Private Chatbot for Your Files, Image Search via Voice | How to get started

  Рет қаралды 8,065

Prompt Engineering

Prompt Engineering

Күн бұрын

This video provides an in-depth review and tutorial of NVIDIA's ChatRTX, a new tool designed for users with RTX GPUs on Windows PCs. The tool leverages Retrieval Augmented Generation technology and tensor RT LLM alongside RTX acceleration to chat with documents and use voice interaction. It now supports local photo and image search with improvements in its features. The application requires specific system configurations including an RTX 30 or 40 series GPU. The video covers installation, model downloading like Mistral 7B and Whisper for speech to text, and showcases the application's ability to retrieve information from documents and images stored locally. It demonstrates the chat functionality, interaction with text and voice, searching capabilities within documents and images, and highlights its potential utility for users with corresponding NVIDIA hardware.
🦾 Discord: / discord
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Patreon: / promptengineering
💼Consulting: calendly.com/engineerprompt/c...
📧 Business Contact: engineerprompt@gmail.com
Become Member: tinyurl.com/y5h28s6h
💻 Pre-configured localGPT VM: bit.ly/localGPT (use Code: PromptEngineering for 50% off).
Signup for Advanced RAG:
tally.so/r/3y9bb0
LINKS:
nvda.ws/3WmXEiy
TIMESTAMPS
00:00 Introduction to NVIDIA ChatRTX
01:16 Setting Up NVIDIA ChatRTX
01:59 Exploring Chat Features with AI Models
04:00 Chatting with Documents Using ChatRTX
06:25 Testing Different AI Models and Image Search
09:04 Final Thoughts and Future Possibilities
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu...

Пікірлер: 37
@engineerprompt
@engineerprompt 25 күн бұрын
If you are interested in learning more about how to build robust RAG applications, check out this course: prompt-s-site.thinkific.com/courses/rag
@im-notai
@im-notai Ай бұрын
Nvidia should also provide an option to select custom model from hugging face as well. So we can always select the latest and best one according to our preference
@engineerprompt
@engineerprompt Ай бұрын
I agree, that would be really helpful
@victorfsaaa
@victorfsaaa Ай бұрын
I am actually surprised that they didnt gave llama 3 as option
@RedVRCC
@RedVRCC Ай бұрын
This is kinda cool even if its a bit limited rn. I really like the idea of being able to run a powerful AI model locally on my PC, making it literally mine to do whatever with and also not sharing any of my data with a big server. I'd love to see how this progresses. I am downloading it as we speak but I'm not too sure whether or not my lowly 3060 will run it well or not.
@neilomalley9887
@neilomalley9887 Ай бұрын
It would have been nice to test this on more than one text file, and also on various files of various type i.e. pdf, pdf with tables, and then to report on strengths and weaknesses.
@engineerprompt
@engineerprompt Ай бұрын
Good point, will need to do that.
@kaistriban
@kaistriban Ай бұрын
good review. thanks. a tool to experiment but I kind of feel that more soon then later is going to be integrated in file manager. maybe the issue is to make this run without needing so much hardware and consequently energy
@engineerprompt
@engineerprompt Ай бұрын
Agree, its a good tool if you have the hardware to run it. I haven't experimented with windows copilot but very similar functionality might be coming there.
@shotelco
@shotelco Ай бұрын
Now we're getting somewhere with AI. I assume ChatRTX is 100% _Secure and Private?_ Some other public LLM(s) are not secretly "learning" from an Individuals local datasets? (Is ChatRTX making external API calls somewhere?) If so, is there an OCR type functionality so ChatRTX can ingest scanned docs? Can agents interact via API with the included LLM's? Can custom LLMs be used, either natively or via API? User owned Security and Privacy makes this a real "Personal Assistant".
@global_dominance
@global_dominance Ай бұрын
Requests go through the company's servers and they are censored. There is no privacy involved.
@vertigoz
@vertigoz Ай бұрын
I hope it will come soon to the 20 line rtx
@irrelevantdata
@irrelevantdata Ай бұрын
When you say it's faster than ollama, do you mean you ran this same model with same parameters on ollama too, and timed it, and it is indeed slower? How much slower?
@engineerprompt
@engineerprompt Ай бұрын
It's running the 4bit quantized model same as the default quantization in ollama. I haven't done quantification I terms of tokens per second since both chartrtx and ollama do not provide that info but based on empirical evidence I "feel" it's faster
@Ginto_O
@Ginto_O Ай бұрын
@@engineerprompt ollama provides that info.
@Ginto_O
@Ginto_O Ай бұрын
@@engineerprompt ollama run --verbose shows you tokens\sec
@adriangpuiu
@adriangpuiu Ай бұрын
its basic rag, and frankly speaking using the lobotomized quantized models makes no sese in using it
@wmsone
@wmsone Ай бұрын
So true I’m looking for a non woke LLM. Which one should I used please ?
@adriangpuiu
@adriangpuiu Ай бұрын
@@wmsone depends on your need man. i usually use the fp16 models , and for tother things im using the Meta-Llama-3-8B-Instruct-correct-pre-tokenizer-and-EOS-token-Q8_0.gguf
@Ginto_O
@Ginto_O Ай бұрын
@@wmsone non woke? what do you mean? you want heavy censored llm?
@Ginto_O
@Ginto_O Ай бұрын
@@wmsone bing is most censored
@hamidmohamadzade1920
@hamidmohamadzade1920 Ай бұрын
how much is the content window ? can we search on a pdf with 80 pages?
@engineerprompt
@engineerprompt Ай бұрын
It's seems to be the first version so I would think it will be 8k
@hobologna
@hobologna Ай бұрын
Trying with 6VRAM on my laptop's RTX 3060
@im-notai
@im-notai Ай бұрын
please update me if that works
@hobologna
@hobologna Ай бұрын
@@im-notai it doesn't unfortunately, won't even allow me to install
@longboardfella5306
@longboardfella5306 Ай бұрын
It doesn’t work on my Quadro RTX8000. I can run Llama 3 models so it’s pretty disappointing that my large VRAM of 48GB and lots of Cuda and Tensor cores is not enabled to run this
@engineerprompt
@engineerprompt Ай бұрын
Seems to only support RTA 30 and 40 series. Hopefully they will add support.
@alx8439
@alx8439 Ай бұрын
I think the speed here is just a function of a small and heavily quantized model and not the TensorRT magic.
@engineerprompt
@engineerprompt Ай бұрын
It could be. Would be interesting if they would let you bring other models
@Ginto_O
@Ginto_O Ай бұрын
How to use llama3 with it?
@engineerprompt
@engineerprompt Ай бұрын
In the demo they show llama3 in the list. Might be coming in a new update, probably.
@michai333
@michai333 Ай бұрын
LM studio is much better, you can also offload all compute to your RTX+ gpu.
@longboardfella5306
@longboardfella5306 Ай бұрын
I’m doing that - is there a model that can examine local text files - I’m a bit new to all this
@bajerra9517
@bajerra9517 Ай бұрын
Nop, i 've tried both and a lot of LLM local ai versions. RTX pretty fast answers than others but the worst thing is you can 🚫 choose your models but Nvidia.
@HassanHammoud
@HassanHammoud Ай бұрын
on of the best options? that is one of the worst ones. it is soooo behind
Yi-1.5: True Apache 2.0 Competitor to LLAMA-3
16:02
Prompt Engineering
Рет қаралды 6 М.
Claude Artifacts: What it can do and limitations
12:37
Prompt Engineering
Рет қаралды 3,3 М.
The day of the sea 🌊 🤣❤️ #demariki
00:22
Demariki
Рет қаралды 78 МЛН
Why You Should Always Help Others ❤️
00:40
Alan Chikin Chow
Рет қаралды 134 МЛН
Ajay Prakash: Innovating Customer-Centric Solutions at Rinse
1:01:39
NVIDIA'S NEW OFFLINE GPT! Chat with RTX | Crash Course Guide
7:54
TroubleChute
Рет қаралды 82 М.
iOS 18 Beta 2 Released - What's New?
15:27
Brandon Butch
Рет қаралды 298 М.
Mesop: Google's Python based UI Framework for Building Web Application
13:42
Nvidia Nim:  Deploy Open Source LLMs with 1 click
4:20
Mosleh Mahamud
Рет қаралды 919
ChatRTX: Introdução e Primeiros Passos na IA da Nvidia
11:17
Marker: This Open-Source Tool will make your PDFs LLM Ready
14:11
Prompt Engineering
Рет қаралды 36 М.
RAG from Scratch in 10 lines Python - No Frameworks Needed!
11:21
Prompt Engineering
Рет қаралды 13 М.
How to Web Scrape in 2024 with JinaAI, FireCrawl, Crawl4AI and More
13:12
Prompt Engineering
Рет қаралды 12 М.
🔥Идеальный чехол для iPhone! 📱 #apple #iphone
0:36
Не шарю!
Рет қаралды 1,3 МЛН
После ввода кода - протирайте панель
0:18
iPhone 12 socket cleaning #fixit
0:30
Tamar DB (mt)
Рет қаралды 49 МЛН
1$ vs 500$ ВИРТУАЛЬНАЯ РЕАЛЬНОСТЬ !
23:20
GoldenBurst
Рет қаралды 660 М.
Gizli Apple Watch Özelliği😱
0:14
Safak Novruz
Рет қаралды 3,5 МЛН
Какой ПК нужен для Escape From Tarkov?
0:48
CompShop Shorts
Рет қаралды 264 М.