"I want Llama3.1 to perform 10x with my private knowledge" - Self learning Local Llama3.1 405B

[@AIJasonZ]

I'm starting my own community to provide more enablement for AI builders, including all in depth code breakdown, expert interview/workshop & more resources (WIP), just getting started! 2ly.link/1z94G

[@lynchmick]

Great video, as always. The link shared above doesn't work for me.

[@amandamate9117]

Jason, is your discord group dead? there is no bot anymore and also some NSFW latina links, very strange.

[@semahselmi]

Appreciate your work jason ❤ i ll join the community asap

[@florentromanet5439]

Great channel Jason. I'll give it a try and have already joined.

[@qwestive4879]

Sick, having an open source alternative to OpenAI tool calling is such important move!

[@connerdimoush4059]

Always clean and simple execution on seemingly complex topics. I appreciate how you utilize different models, tools, and services between the videos. Keep the content coming!!!

[@TheInternalNet]

This is so so so amazing. I'm so excited to be able to have my local bots running in different public social platforms

[@Jim-ey3ry]

Thanks so much, I've been trying to build agents with llama3.1 and couldn't find any; This is awesome! Also gave me a good understanding of how tool calling works in general

[@remsee1608]

It's not smart enough to do it. 405b might be

[@aiplusi]

Nice work! Thanks for putting all these great tutorials together!

[@Joe-bp5mo]

Woahh, they are working on llama4 already? such a ninja move

[@TheFocusedCoder]

Awesome vid, like the use of an orchestrator!

[@abiolasamuel8092]

I clicked on this link because you mentioned "Self learning Local Llama3.1 405B" in your title. But its actually a 4.8B RAG pipeline.

[@henriquematias1986]

What’s the difference? Explain me like i’m a 5 year old

[@abiolasamuel8092]

@@henriquematias1986 what's the difference? Really?

[@AIJasonZ]

Ah I did a simple implementation of you watch till end as it will constantly update knowledge base to learn from the new data provided :)

[@heelercs]

@@AIJasonZthat doesn’t address the concern.

[@heelercs]

⁠@@henriquematias1986405 billion parameters vs 4.8 billion parameters in the core model. Generally, more parameters = better model.

[@jasonfinance]

Gold as always, I also tried llama3.1 tool calling, the result seem not that great still, feel like i haven't setup properly, the prompt example you shown was useful!

[@AjarnSpencer]

The art is indeed in the system prompt as it is equally in the user prompt. I have just created a bio tool and a personality version tool with a personality switcher for llama to add to system prompt. GPT chat has four different personality versions mine has 16. with various sub categories. You can also mix personalities which gives you different results and I can call these personalities when I want the AI to respond in different ways be it succinctly, long winded, contrary, agreeable, empathetic, straightforward, the entrepreneur, the player, the role player, the empathizer, the advisor, ... you can mix the traits and the attributes of different personalities by telling the AI which ones to use, you can empower the AI to switch itself according to what the user is talking about, or allow the user to define which personality he wants to interact with.. Here is a snippet 1. **INTJ (The Architect)** - **Traits**: Strategic, independent, visionary. - **Approach**: Provide detailed, strategic responses with a focus on long-term goals and efficiency. Use analytical and well-structured communication. - **Subdivisions**: - **Strategic**: Emphasize planning and future-oriented thinking. - **Analytical**: Focus on data and logical reasoning. 2. **ESFP (The Entertainer)** - **Traits**: Sociable, spontaneous, playful. - **Approach**: Engage with lively, enthusiastic conversation. Use a friendly tone and include humor or interesting facts. - **Subdivisions**: - **Sociable**: Foster a warm, engaging interaction. - **Spontaneous**: Respond with creativity and adaptability. 3. **INFP (The Mediator)** - **Traits**: Idealistic, empathetic, reflective. - **Approach**: Offer compassionate and understanding responses. Focus on values, personal growth, and emotional support. - **Subdivisions**: - **Compassionate**: Provide empathetic and supportive advice. - **Reflective**: Engage in deep, thoughtful discussion. 4. **ESTJ (The Executive)** - **Traits**: Practical, organized, assertive. - **Approach**: Give clear, practical advice with a focus on structure and efficiency. Use direct and straightforward communication. - **Subdivisions**: - **Practical**: Offer actionable, real-world solutions. - **Organized**: Maintain clarity and structure in responses. 5. **ENTP (The Debater)** - **Traits**: Innovative, outgoing, analytical. - **Approach**: Engage in dynamic, argumentative discussions. Encourage exploration of new ideas and challenge existing perspectives. - **Subdivisions**: - **Innovative**: Propose novel ideas and solutions. - **Analytical**: Examine and debate various viewpoints.

[@AIMLDemo]

this will transform lot of industry!

[@AIMLDemo]

Informative !!

[@sadmansakib007]

Hey Jason, Thanks for such a wonderful walk-through! I have a question: is it do-able with discord instead of slack?

[@ChasingDream2002]

I new to the AI train but, would it be better if we can organize our functions and provide docs to our LLM models like RAG then those AI can evaluate which function to call base on the context? Of course it would need a large amount of well crafted docs with category, purpose, related contents so the LLM can know which docs to look through and execute the function, depend on the request, we can let the function use the internet to perform online services instead of letting the AI accessing internet data

[@WirelessGus]

Thanks for sharing Jason! Great Knowledge 📖 share 🤝

[@amandamate9117]

Jason, is your discord group dead? there is no bot anymore and also some NSFW latina links, very strange.

[@LloydBettellHiggins]

I genuinely love what you are doing but why would you not use something like Flowise locally with Llama3.1 locally as well?

[@florentromanet5439]

Great work and thanks for sharing 😊

[@Arvolve]

Very cool!

[@proflead]

Thanks for the great content!

[@thisisvaze]

Amazing videos. Thanks a lot!

[@JoshuaGottlieb-oz4er]

Amazing content, thank you

[@frankdearr2772]

great topic, thanks 👍

[@ingridw3900]

How do we feel about putting our personal agent in the hands of the company? What happens if we leave? Who owns our Future productivity?

[@seanolivas9148]

Is the API not private like openai?

[@coolmcdude]

I’ve been trying to do this but with discord.

[@kemil7885]

There is no more llama index right ?

[@glaudiston]

Thank you for sharing. But can we really call this "local"? I mean... we are sharing and trusting external resources and endpoints. Not something we can really use in a private environment.

[@marcc0183]

Anyone knows how to do this??? 7:10. I think that in that case it would be better to do fine tuning to imitate the speaking style, tone, personality, etc

[@zacharyhall7466]

invasion of the body snatchers

[@seanolivas9148]

What's the context window here?

[@aaagaming2023]

How is deepseek code v2 at tool calling?

[@titusblair]

@AIJasonZ I don't see index in LlamaCloud is this a certain level of access? Thanks!

[@TemporallyDifferent]

Same and I thought I’m going crazy. @AIJasonZ Thoughts?

[@AIJasonZ]

It is in private beta, but feel free to drop me your email or join my community link and will get you access!

[@global.pradachan]

we want more videos on llama agentic system integration with the llama llm

[@michimansi]

if a learning is incorrect because somebody made a mistake is it possible to change it

[@3barazi1]

great ! keep it up

[@andreaserrano3809]

Hey, you are pulling llama3.1 from ollama and it only takes 4gb so it must be the 8b version. Am I correct?

[@AIJasonZ]

Yes that correct 8b model!

[@makeoutmonday3249]

Hey, I recently discovered your videos and I'm really fascinated. I binged watched like 5 videos today and I think I learned a lot. I'm also interested in joining your community but as a student 27$ a month is really rough. You totally earn it and you also provided a lot of knowledge for free, but isn't there a way to maybe lower the price or at least add other options? Thanks in advance

[@amandamate9117]

yeah like 9$/month would be nice

[@mf4552]

Is it actly 27

[@anuragshas]

Since when RAG became a method of training a model?

[@doctorkiro]

Jason, if I try to use a language that's not English, in this case Brazillian Portuguese, will the model still perform well?

[@theskiesaredark]

It should work out of the box, because LLMs were initially used for translation. You basically can get free i18n, let's say your knowledgebase is in English, it can auto-translate to Brazilian Portuguese. Or vice versa (KB is in Portuguese, user query is in English).

[@doctorkiro]

@theskiesaredark Thanks🫡

[@AjarnSpencer]

i've already got it on my PC but I need to get a more powerful PC because the CPU and the graphics unit is too slow. Takes an hour to produce 300 words. I'm just looking at some desktop computers or maybe build my own, but with GPT for all open source and you can download lama from Meta which is now open source and mistral orca, and falcon and many other models many of which have commercial license or you can just use them for personal use like I do without any involvement with third parties.

[@kerron_]

I like tool calling

[@nicoschmitt9857]

imagine they didn't use 1/3 of the compute to censor, bias and restrict answers from the main model...

[@flrn84791]

Where's the 405B?

[@seregal7739]

why not to use memGPT?

[@leononymous2562]

Why is there still no easy .exe file for a personal AI Agent for normal people? Who do not know anything about this topic?

[@mhwebdev]

Take a look at llamafile

[@pra8495]

github link please

[@travelrealindia1]

Please share the github repo

[@ahmedalshareef7168]

I just got enlightened..

[@Codewello]

Among all the companies in the world, Meta is the one being the most open-source-unlike a certain company called OpenAI. 😂

[@MyrLin8]

Private; LMAO. Thanks anyway 'Zuck', Kudos for the release :)

[@lucasvilela2291]

excelent

[@Larimuss]

"How to connect to CRM" yeh that would make a good tutorial video. Maybe to azure 365 dynamics 🤣 I guess you can just connect straight to the SQL Database and use darafactory to pipe the data to a more usable format database for vectoring

[@MrAtomUniverse]

At 8:00 , that's the fastest way to lose your job lol

[@luisbarbosa5597]

nice

[@CristianAguilarnavarro]

Thank

[@spa0220]

Nice kitchen, I would like to be invited for a hot pot dinner with you!

[@SebKrogh]

Llama 3.1 405b with Agent Zero 🙃

[@davidhardy3074]

Sorry ill keep looking till I find a video that can make llama3.1 perform 100x and it knows things even I dont know about myself.

[@devagarwal3250]

pls dont add shitty genai frameworks

[@teachwj]

facebook is lame

[@guylast9516]

Thanks for the video. Getting this in Win 11 in terminal window so stuck at start. Not sure what I am doing wrong PS C:\Users\joe> ollama run llama3.1:8b 2024/08/06 19:27:17 config.go:47: WARN invalid port, using default port=\Users\joe\AppData\Local\Programs\Ollama\ollama.exe default=11434 2024/08/06 19:27:17 config.go:47: WARN invalid port, using default port=\Users\joe\AppData\Local\Programs\Ollama\ollama.exe default=11434 Error: Head "C:11434/": dial tcp: lookup C: no such host PS C:\Users\joe>