3 ай бұрын
Пікірлер
@Matt-rq6wz 13 сағат бұрын
Am I understanding correctly, that the prompt can't be passed from Ollama WebUI into Comfy? Meaning, whatever prompt I set in comfy is the image I get in WebUI. In your example, 'a pretty young lady' can't be changed from the WebUI?
@MukulTripathi 12 сағат бұрын
You can edit the generated output by model and then that becomes new prompt and then click on regenerate button again.
@jesusleguiza77 Күн бұрын
Hi, how can I get advice? Do you have an email address? Thanks
@balajiraghavan 2 күн бұрын
I got it through a back order on Sparkfun; ordered on 12/17; shipped on 12/27.
@BKHANEJA1 3 күн бұрын
How can we keep a prefect server running persistently? also where it store its transaction logs ? Any database ? Also how to keep workers running? Sl
@julianvanderkraats408 7 күн бұрын
Oeee thank you! Have it running and can totally recommend this setup and this supercute little pet robot!
@MukulTripathi 7 күн бұрын
💯 it's just perfect! I'm going to make custom intents to make this even fun!
@zerosequels 7 күн бұрын
I just installed comfyui but I can't find the side bar, it's not coming up
@MukulTripathi 7 күн бұрын
I think the UI has changed since I made the video. I made an updated video for Jetson. Please check this out for newer UI. kzbin.info/www/bejne/sKmQp5ybgpWHp6M
@k6fi585 7 күн бұрын
how can I validate that searxng is using the valkey/redis instance?
@vijaypurbia 7 күн бұрын
Is there any way to stop/skip a flow run based on some condition? like if we are not getting any data in task then i want to skip the flow run(don't want to execute the other tasks.)
@rsilveira79 10 күн бұрын
Really nice video, thanks 🙏
@MukulTripathi 10 күн бұрын
You're welcome!
@renanmonteirobarbosa8129 12 күн бұрын
Advice, make sure the SSD is properly formated or it will mess with the bootloader
@itachihd463 12 күн бұрын
followed your steps, but I add GFX at step 1, and there error on host machine when install Graph Composer🥲
@akashgeorge5433 12 күн бұрын
how much did it cost you to get orin nano in india?
@MukulTripathi 12 күн бұрын
I'm not sure! I purchased it from AliExpress.
@kaushalpatel6709 13 күн бұрын
is wifi working in your case ? i mean are you able to connect to wifi network from the list ?
@MukulTripathi 13 күн бұрын
Yes I am. Wifi works as normally it does on Ubuntu 22.04.
@kaushalpatel6709 13 күн бұрын
@@MukulTripathi Thanks.
@djmotion86 16 күн бұрын
The music is so annoying, gosh
@MukulTripathi 16 күн бұрын
I got this feedback from many viewers. And I apologize for making the music so prominent in there. I'll release a version of this video with no music as well and in future I'll make sure not to have that prominent music in there. I really appreciate your feedback though. Thank you for it.
@mabinonline 16 күн бұрын
This is really excellent content with explanation. Great work
@MukulTripathi 16 күн бұрын
I'm glad you liked it.
@gf3803 17 күн бұрын
Loving your video although the background music makes it tough to hear you....
@MukulTripathi 17 күн бұрын
Noted! I'll make it subtle next time around. And thank you!
@nickthompson2052 18 күн бұрын
Subscribed, great content... excellent delivery
@MukulTripathi 18 күн бұрын
Thank you! :)
@GermanPaak 18 күн бұрын
Good work
@MukulTripathi 18 күн бұрын
I appreciate it!
@TheOhamd 18 күн бұрын
this Music in the background sucks
@MukulTripathi 18 күн бұрын
Dang! I thought it sounded good! I'll take your feedback and make it more subtle. Thank you for the feedback, I appreciate it.
@energyideas 18 күн бұрын
This video demonstrations expertise in AI, lots of hard work getting the end positive result. That being setting up a free, offline image generator on Jetson Nano Orin with customization for what you want and don't want in the image. Thanks so much for your work on this video.
@MukulTripathi 18 күн бұрын
Thank you! I appreciate the kind words!
@stephencurrie2710 18 күн бұрын
Hello Mukul, I greatly appreciate your videos! I have the same version of Jetson Orin Nano but I am having issues with flashing the jetpack 5 onto the SD card. I have tried several times to get it to work with no luck. After I flash the SD card with Etcher, the SD card it is no longer readable and it disappears. Are there any steps that I need to take to prepare the SD card before flashing?
@MukulTripathi 18 күн бұрын
Once you flash the SD card with etcher, have etcher verify it too. If verification process goes through but you cannot see the contents of SD card in Windows, it's ok. Just continue inserting the SD card in Jetson and continue from there. If SD card doesn't work after that, then maybe try a different SD card. Flash and initial boot part of process is the easiest one, more challenges will be coming down the line :)
@jaspalsingh908 20 күн бұрын
Which jetson orion should I go with? I'm an ai Engineer who wants to try local llms
@MukulTripathi 20 күн бұрын
Depends upon the requirements, this is cheap and good alternative for edge deployments. You can build an AI server instead with a 30 series card (3060-3090) for about 500-1000 with used parts. I also have a few video guides if you wanna build your own Linux server for AI.
@energyideas 22 күн бұрын
Jetson Nano Orin Super 8gb: Running ollama llama3.2:3b on a stand alone, no wifi, no internet, based on your instructions is mind boggling. You can ask it anything and get an answer. How the heck is this possible from such a small form factor? From neuroscience to cdte photovoltaics, it gets it right. Remember, the dev kit is not doing the computations. Thanks @mukul_tripathi.
@luisonelio70 22 күн бұрын
Can I use a virtual machine with Ubuntu installed?
@MukulTripathi 22 күн бұрын
I think in the comments below Julian has tried it out with a VM. You should see that comment.
@JoshFisher567 13 күн бұрын
In Nvidia's documentation it's HIGHLY recommended not to use a VM of any kind. I used a Ubuntu 22.04 bootable USB drive. No issues. Just don't try to install everything or I had the download location wrong as the USB drive failed due to lack of space so I'm dual booting Ubuntu 24 and 22 now with 22 on a 30GB partition using GRUB.
@MukulTripathi 13 күн бұрын
Thanks good to know Josh.
@jesusleguiza77 22 күн бұрын
The TTS looks very fast, that's good. Is there a way to train it to use Spanish with a Latin American accent? Regards.
@Skandar0007 22 күн бұрын
The window at 12:50 is not showing to me (to select SSD storage device), I don't know why.
@Skandar0007 22 күн бұрын
Figured out the issue, Remove the SD card when you want to flash to the SSD
@jesusleguiza77 22 күн бұрын
Excellent content! 1. I'm wondering if a high-end CPU like the i7-13700K could match the token speed in this case with the 7B model. It would be interesting to make a similar comparison. 2. Is the call function quality with the 7B model comparable to that of the 32B model? Regards
@MukulTripathi 22 күн бұрын
Take a look at this video. I created a tool for this exact purpose! kzbin.info/www/bejne/onPdmYiilJaVj9E
@jesusleguiza77 22 күн бұрын
@ Thanks!
@michelamar-khodja8591 22 күн бұрын
Jetson Orin Nano Developer Kit vs Jetson Orin Nano Super Developer Kit. Jetson Orin Nano Developer Kit : GPU NVIDIA Ampere avec cœurs Tensor CPU Arm à 6 cœurs Capacité de traitement neuronal de 40 TOPS (trillions d'opérations par seconde)1 Bande passante mémoire de 68 Go/s. Jetson Orin Nano Super Developer Kit : GPU NVIDIA Ampere avec cœurs Tensor CPU Arm à 6 cœurs Capacité de traitement neuronal améliorée de 67 TOPS Bande passante mémoire de 102 Go/s Inclut également une carte porteuse de référence et un module système-sur-puce (SoM) Jetson Orin Nano 8 Go. Logiciel mis à jour pour augmenter les fréquences d'horloge du GPU, de la mémoire et du CPU. En résumé, bien que les deux kits partagent beaucoup de similarités matérielles, le Jetson Orin Nano Super Developer Kit offre des performances améliorées grâce à des mises à jour logicielles et des capacités de traitement accrues. Le Super Developer Kit inclut en effet des améliorations matérielles supplémentaires, à savoir la carte porteuse de référence et le module système-sur-puce (SoM) Jetson Orin Nano 8 Go, qui ne peuvent être obtenus par une simple mise à jour du logiciel. La bande passante mémoire de 102 Go/s est une amélioration matérielle spécifique au Jetson Orin Nano Super Developer Kit. C'est une augmentation significative par rapport aux 68 Go/s du Jetson Orin Nano Developer Kit. Toutefois, atteindre cette bande passante mémoire accrue nécessite d'utiliser les composants supplémentaires fournis avec le Super Developer Kit (comme la carte porteuse de référence et le module système-sur-puce Jetson Orin Nano 8 Go).
@ronwiltgen2698 23 күн бұрын
I purchased this after watching your video. It's sold out indeed, but someone in local FB marketplace was selling it for $400. I guess it was okay. Now time to go through this guide and install everything!
@MukulTripathi 23 күн бұрын
Yeah, it's sad but even $400 is a deal on these at this time. You got lucky.
@polysopher 23 күн бұрын
It's not working!! Can't get to ubuntu screen rebooting on ssd with 5 Jetson version.
@MukulTripathi 23 күн бұрын
What step of the process are you in? Were you able to upgrade your firmware?
@polysopher 23 күн бұрын
@ no! When I get the ssd it goes black after few seconds. It kills the system
@MukulTripathi 23 күн бұрын
This video doesn't talk about SSD (solid state drive), in this video we use micro-SD card to update the firmware all the way to 36.x Only one your firmware is upgraded, then (in the next video) in the series we use the SSD.
@polysopher 23 күн бұрын
@@MukulTripathi No no a typo I'm using the same micro sd card. Did you modify any settings in the boot loader options?
@MukulTripathi 23 күн бұрын
No I didn't have to modify any settings. It's designed to boot from SD card by default. Try flashing the 5.x image and if that doesn't work, it's worth trying to flash 6.x image and check. It's possible that your firmware was already upgraded and that's why it's not booting with 5.x jetpack. Only a speculation but it's worth trying.
@julianvanderkraats408 23 күн бұрын
The process is failing for me at the flashing stage, I guess because I am using a virtual machine, and the machine is not re-added automatically after it is removed (the USB connection). Or maybe some other reason?
@MukulTripathi 23 күн бұрын
These two things will help: 1. Use a high quality USB type c cable. 2. Plug it directly into the motherboard. I noticed that a USB hub and not a high quality USB cable fails sometimes. Happened to me too with a hub.
@julianvanderkraats408 22 күн бұрын
@@MukulTripathi Thx. Turns out a bunch of things were wrong. A) Virtualbox does not work well (hangs on 99%), where VMWare Player does work. B) had to use a different USB port C) It only worked when I did unplug the jumper during flashing, as instructed by NVIDIA for the "manual" proces - so the flashing of the NVMe drive took place in normal mode, not in recovery mode. That does mean that during the process a different device will also need to be coupled to the VM, so to avoid that you first connect it in normal mode to get that out of the way. Also a couple of other tips for people going the same way as me: only use Ubuntu. Give the machine at least 8GB or else SDK manager won't start. Set the resolution (in the VM itself, so in Ubuntu) to fullHD or more, or again SDK manager won't start. And give the machine 50GB of HD space, or else you will run out of space (depending on your choices).
@energyideas 23 күн бұрын
Great tutorials on how to set up 3B and 7B LLM's using SSH. Thanks so much. Love your check lists. Keep up the great focus.
@MukulTripathi 23 күн бұрын
I appreciate the kind words, thank you!
@julianvanderkraats408 24 күн бұрын
Ha I see you use a Vector home there! I bought the Jetson solely to be the new brains of my Vector! If you have had any success with that by any chance let me know!
@MukulTripathi 24 күн бұрын
I'll be doing that next! There will be a new tutorial on it soon :) Checkout vector's discord, I'll be posting it there as well. I'll be doing some development on wirepod as well to bring more local AI to it.
@oorischubert4085 24 күн бұрын
Awesome thanks
@MukulTripathi 24 күн бұрын
Welcome 😊
@AmanKumar-jk1qu 24 күн бұрын
Great tutorial !!
@MukulTripathi 24 күн бұрын
Glad you liked it!
@GeorgeDekker 24 күн бұрын
thanks! some great concepts in your code 🙂
@MukulTripathi 24 күн бұрын
You're welcome!
@dvinemuteta7674 25 күн бұрын
How's the power consumption on this server
@MukulTripathi 25 күн бұрын
It's not easy on power. About 150-200 watts on average. It goes up to 600-700 watts on peak consumption.
@dvinemuteta7674 25 күн бұрын
@@MukulTripathi Many thanks for the prompt feedback well I'm stuck perhaps you can assist I'm a cyber security engineer and would like to enhance my skills through labs been researching all day 80% of my findings are leaning towards 730xd do you perhaps have any recommendations for an affordable home lab server which does not consume power, efficient
@MukulTripathi 25 күн бұрын
@@dvinemuteta7674 take a look at my server build playlist. There's a 2x3090 setup and a 4x3090 setup for better idle watts.
@dvinemuteta7674 25 күн бұрын
@@MukulTripathi Many thanks will check it out end game for me is to buy something for longevity 5- 10 years sort of like an investment
@MukulTripathi 25 күн бұрын
@@dvinemuteta7674 a 3090 will be a good investment! Definitely take a look at hardware listed in my 4X3090 that will take you to next 5 years easily.
@WisherTheKing 26 күн бұрын
I have the original version of this. I wonder if it’s possible to make it work
@MukulTripathi 25 күн бұрын
It'll work with any Vector.
@___Truth___ 26 күн бұрын
Looks really fun😂 I Gotta buy and play🎉
@MukulTripathi 26 күн бұрын
It definitely is a lot of fun!
@piymimi 27 күн бұрын
Love it
@piymimi 27 күн бұрын
Wow
@shekharmanna3303 29 күн бұрын
Wonderful explanation!!
@MukulTripathi 29 күн бұрын
I'm glad you liked it!
@jesusleguiza77 Ай бұрын
If I only have one 3090 GPU, how can I use local RAM even if that makes the inference slower? Regards
@MukulTripathi Ай бұрын
In Ollama docker setup you can specify it. If you see my AI Server Setup video, in the plans command I'm specifically talking about the GPU in the command. If you skip the GPU related steps or specify CPU, it'll fallback to CPU.
@jesusleguiza77 Ай бұрын
@@MukulTripathi But can I use GPU and CPU at the same time? For example, to use a model with 16GB of GPU VRAM and 10GB of CPU RAM, or would it only load into the CPU?
@MukulTripathi Ай бұрын
I think you can use llama.cpp and offload some layers to GPU and remaining to CPU. I didn't explore that option much but it's possible.
@jesusleguiza77 Ай бұрын
@@MukulTripathi so that would be directly with llama.cpp and no longer with ollama, right?
@MukulTripathi Ай бұрын
That's accurate. If there's a way to do split inference (GPU+CPU) with Ollama, then I'm not aware of it.
@jesusleguiza77 Ай бұрын
same good i7-12700k?
@MukulTripathi Ай бұрын
Actually this machine had i9-9900K
@exciteresearch Ай бұрын
Yes! Exactly what we need! 🎉
@ivideogameboss Ай бұрын
6:21 OMG my jaw dropped when you said that. Thats crazy! Now I know for sure I am poor.
@MukulTripathi Ай бұрын
I have a total of 7 3090s, some P40s, some 3070s eand some more that I don't recall. I collected them over a period of time. :)
@MukulTripathi Ай бұрын
The __consumer_offset is an internal topic not a table :) i think i said it as a table a couple of times in the video.
@exciteresearch Ай бұрын
Really wonderful! 🎉
@MukulTripathi Ай бұрын
Glad you liked it!
@mukulsingla2070 Ай бұрын
@MukulTripathi Can it be possible to sync up with you? I am AI enthusiast with 15 years of experience.
@ronwiltgen2698 Ай бұрын
Right around the 17:18 mark, there is a blank video for 10-15 seconds.