Installing KoboldCPP on Windows

[@ghoulsgarden]

Finally, the holy algorithm has taken pity on me and brought me something useful! I've been playing around with LLM since the beginning of May and trying out some advanced stuff. To be honest, I still don't know what I'm doing most of the time. First I used Kobold Ai United, then TextGen WebUI and now I want to go back to Kobold, but I don't understand the GUFF format. (Sometimes I have a bit of trouble because English isn't my native language) Thanks for a video that my constantly distracted butter brain can finally follow and understand 🤩 Other explanations were too dry. Keep up the good work! I hope you get more subscribers.

[@cognibuild]

glad to hear it! KCPP is the holy grail of LLM. You dont need to understand gguf other than that those are the files you need. Come over to the Discord channel: discord.gg/MXHZuAcP and I can help you find more models

[@lilkrabs]

God, thank you so much! I was SO confused! I was struggling for an hour trying to get this installed. :(

[@cognibuild]

glad you figured it out! KCPP is awesome and pretty much the only chat loader i use. WHen you get it going and feel comfortable, watch the vids on how to install Llava, a micropjhone, and a text-to-speech server :'D hahah .. i also have a quick installer that comes with some-preloaded models over at www.patreon.com/cognibuild if you need further help

[@lilkrabs]

@@cognibuild Funny enough, now I can't figure out how to run it on sillytavern lol

[@cognibuild]

@@lilkrabs i havent really played around wiht ST although I should. I'll load it up and see what I can find. Come over to discord and we can chat easier there: discord.gg/JfWJXXUu

[@mavericks2k811]

Lol that hammer analogy is perfect I was using perplixity ai and so frustrating thanks for your videos

[@cognibuild]

Glad to help and appreaciate the feedback! Let me know if theres anything else that would be helpful

[@TheSilverazo]

thank you so much, I appreciate how you explain everything too

[@TPkarov]

Nice tutorial !

[@iknowwhy2629]

Yo thanks for the video.

[@cognibuild]

of course. Let me know if theres other stuff you want to know

[@NineLayerNige]

Hello there. Thank you, I managed to follow your tutorial and successfully installed all components and have been able run KoboldCPP. I have an RTX4080Super with 16GB vram and set the Context Size slider to 16k and it ran nice and fast. 24k was quite sluggish in making up the chat. The chat model was good, I think. Anyway, I have a question...you recommended capybarahermes-2.5-mistral-7b-gguf which works fine but being a total newbie to LLM and AI, the github repository is well confusing to navigate and find other suitable models to download as I dont see any description as to what the vast range of modlsl do or their purpose and which are compatible with KoboldCPP. I know you said that any GGUF file will run, but how can I filter the library list or see a description of what the models are for. Do you have a "known to work and great roleplay/fun chat" list or do I just pick ones at random and luck dip ? thanks for your great work on this.

[@cognibuild]

i really like Poppy Porpoise huggingface.co/Lewdiculous/Poppy_Porpoise-0.72-L3-8B-GGUF-IQ-Imatrix/tree/main but yeah, a lot of it is finding model creators that make good stuff. Look up Lunaris as that is one that people have been excited about. Iv ebeen playing with Blackroot Grand Horror, which is kind of interesting because (beyond its macabre nature) uses interesting wordage. The beautiful thing about KCPP is that you can really just try whaever model you want quickly.. if its not what youre looking for, just delete and get another :)

[@cognibuild]

come over to our Discord and we can chat a bit more about cool models discord.gg/jP9hGfUH