The AMD 7900 XT is supported by Ollama "out of the box". No need to use the modified version. But you might also want to make sure your OS and drivers are up to date.

Good deal! Glad to hear it!

Glad to hear it! Best of luck with your project!

Like with the AMD APUs? I'm not sure. Even if it was supported, I doubt if it could perform very well. Too many limitations.

Very cool! Thanks for the feedback.

Great! Thanks for the feedback!

You're welcome! Glad it worked!

Good deal. You're welcome!

Does this help? To download models, navigate to the Ollama Settings in the Connections tab. Alternatively, you can also download models directly by typing a command like ollama run hf.co/[model creator]/[model name] in the model selection dropdown. This action will create a button labeled "Pull [Model Name]" for downloading. docs.openwebui.com/features/workspace/models#model-discovery-and-importexport

Hi, did you install Ollama and pull some models? If so, they should then show up in Open WebUI. Make sure Ollama is running and working first.

Hi, you will need to determine the LLVM target of your GPU and then use the related ROCBLAS packages. I would recommend looking at your log file and determine that target which should start with gfx. In your case it might be GFX1033 but I'm not 100% sure. Maybe someone else can chime in here. Here is the place in the video that will show you in the Ollama log file: kzbin.info/www/bejne/fV7Ooamiq7CAZ8k Here is the latest release (currently) of the ROCBLAS packages: github.com/likelovewant/ROCmLibs-for-gfx1103-AMD780M-APU/releases/tag/v0.6.1.2 Here is a video I made to show how to look at the log file: kzbin.info/www/bejne/pZ-Wo6evhbyGnaM Hope that helps!

Sure, I was simply using the feature built into Microsoft Edge, where you can highlight text and have it read to you. Thanks for watching!

Hey, thanks for the feedback! I appreciate it.

You're welcome! Thanks for the feedback!

Glad it helped! Thanks for the feedback!

Yep, I do remember, now that you mention it. Glad it helped. Llama 3.2 3B worked well for me. Mini models like Phi also work well on my system and might be worth a look. I think if you try models with that smaller parameter count you should see the improved speed over the larger models (7 or 8B, etc...). Which Obsidian plugin are you using? ollama.com/search

Thanks, will do!

Otherwise Great video, thanks a lot.

I have run into this as well. You might try a smaller model (3B parameter like Llama 3.2 or maybe Phi). I'm wondering if it's a bottleneck between transferring data between the CPU and GPU and it could be a matter of needing more RAM. I address this in my video on performance: kzbin.info/www/bejne/omHXlGWKiN2ehZo

Thanks for your feedback!

It should work. That is an LLVM target of gfx1032. rocm.docs.amd.com/projects/install-on-windows/en/latest/reference/system-requirements.html#supported-gpus-win github.com/likelovewant/ollama-for-amd#windows

Yes you can. Follow the steps

Thanks for watching! And for your feedback!

Good deal! Thanks for your feedback!

Thanks, I'm glad it was helpful. The problem you are having usually points to a resource problem. Switching to smaller models like Llama 3.2 3B (or models that are around that size) might work better. Just keep in mind it's a trade off because you do lose some quality. If you haven't yet watched my video on performance, you might want to check it out as it demonstrates a similar problem I had when using Llama 3 on one of my systems. kzbin.info/www/bejne/omHXlGWKiN2ehZo

Great! I'm glad it worked for you. I also have videos on Docker and Web UI if you need help with that. Thanks for watching!

Hello and thanks for watching. I'm not sure, but in a chat situation, I would think it could be because it is storing previous prompts in memory in order to "remember" the previous conversations. How much it keeps could depend on the context window of the model. I have not tested this but that's my best guess. Someone else may have a better answer.

@@TigerTriangleTech Thanks for your reply to my question, Hope you can solve it later😁

Thank you! And great question. I show my specs (for the computer I mainly use for recording) and address how to get better performance with Ollama in my 3 latest videos. Here is one of them... kzbin.info/www/bejne/omHXlGWKiN2ehZo I hope it helps!

I'm glad to hear it. Thanks for watching!

Hi there! Actually, the video I made was for a discrete GPU, but it was AMD rather than NVIDIA. You shouldn't have to install any work around. Ollama supports Nvidia GPUs with compute capability 5.0+. You should be in good shape because the GTX 1050 has a compute capability of 6.1. If this is not working correctly you might want to update your drivers? Not sure. To force CPU usage, set an environment variable of CUDA_VISIBLE_DEVICES to -1. To enable GPU usage just remove that variable and it should utilize your GPU again. For more info view this page: github.com/ollama/ollama/blob/main/docs/gpu.md

Great! Thank you for watching and for your feedback!

Hi there! Thanks for your patience. I have been busy on other projects lately. Open WebUI actually sits on top of something like Ollama so it would not have an API key that you would use. Understanding API Keys: Open AI API Key: You need this if you're connecting to OpenAI's services. This isn't related to running local models but is for accessing their cloud-based models. No API Key for Local Models: When using Ollama or similar local AI runners, you generally don't need an API key for the model itself since it runs on your machine. Exposing Local AI to External Apps: Local AI Access: If you want to access Ollama or any local AI from outside your local network (like in VR, on your phone, or through a Discord bot), you'll indeed need to make your local service accessible over the internet. Here's how: Dynamic DNS Services: Services like No-IP can give your dynamic IP a static hostname. This is useful but might not directly help with API access unless paired with port forwarding or tunneling. Tunneling with ngrok: This is a great suggestion for exposing a local server to the internet temporarily. However, for a more permanent solution, you might look into setting up a reverse proxy or using a VPN. Securing Your Service: If you decide to make your local AI accessible via an API (which would be necessary for integration into apps or bots), you should indeed secure it: Create an API Endpoint: Your app or bot would interact with this endpoint. Here, you might consider generating a "secret API key" not for Ollama but for your application layer that interacts with Ollama or another local AI. This key would be used to authenticate and authorize requests to your service, not to interact with the AI model itself. Security: Use HTTPS for your API, implement authentication (like API keys or OAuth), and consider rate limiting to protect your local resources. Implementation Ideas: For Discord Bot or Mobile App: You would write an application that communicates with your local AI through a secure API you've set up. This app would use your custom API key to authenticate its requests to your local server. Future Video Content: This is indeed a fantastic idea for a video or a series! Covering how to set up a local AI, expose it safely to the internet, secure it with API keys, and then integrate it into various platforms could be very educational. Thanks for watching!

Very true! Hey, thanks for watching!

Thank you for watching! I believe these open models, especially the uncensored ones, ran locally have a bright future and the people that are learning about them now will be ahead of the game.

Thanks Cage-a-liciousAI! And keep making those N. Cage videos. Very creative!

Yeah, running these locally is going to depend on your local resources. You will definitely want a good GPU and Ollama should take advantage of that. Still can be a bit slow though it seems. Also depends on the parameter size of the model. Thanks for watching!

If you haven't yet watched my video on performance, you might want to check it out as it demonstrates a similar problem I had on one of my systems. kzbin.info/www/bejne/omHXlGWKiN2ehZo

Glad it was helpful! Thanks for the feedback!

I'm really glad to hear that! Thanks for letting me know.

So far so good :)

Yep, more cards to keep track of, so that will be interesting.

@@TigerTriangleTech can grok really shuffle a deck and not know what the cards are ? and with poker you need to figure a way for there to be some skin in the game something to win or lose, do a bit longer game too,,

​@@hugokappes4077 The honest answer to that is, it depends. For a few reasons: First, this is generative AI and not specifically trained for poker so the way cards are tracked, and score is kept will differ especially since there are so many variations of the game. Second, Grok is an AI chatbot as opposed to a rules-based chatbot, so it's non-deterministic. Which means the output is not always the same and will vary from session to session. And finally, AI chatbots can still give wrong answers but do it with confidence (known as hallucinations). Thank you for watching and thanks for your feedback! I appreciate it!

Hello again!

Thanks, it was fun to put together. Time consuming but fun 😃.

Thank you!

@@TigerTriangleTech looking forward to the next one,, i may make some suggestions when i thin of something,,

@hugokappes4077 sounds great! Would love the feedback. Yeah, I have some stuff in the works.