You want learn RAG beyond basics? Make sure to sign up here: tally.so/r/3y9bb0
@VerdonTrigance3 ай бұрын
You said about phi-3 small, but it's not releazed yet. Later in the video you are downloading phi-3 mini, which smaller than small.
@VerdonTrigance3 ай бұрын
I'm actually wait for phi-3 small 128 k context length for talking to the documents which are set of different docs like docx, xlsx, txt and python scripts. They are all relevant and I want to put them all in a RAG, but maybe routing will be helpful for that too. But I need a really big context for that. Or I should somehow train it. Only option I know to train on a big document set is ask another model to generate questions and later ask to answer that questions. Anyway, any of thse would be helpful.
@PseudoProphet3 ай бұрын
It's not making any mistake, meta is the real open AI. 😂😂😂
@3choff3 ай бұрын
Brilliant content! I think it is more interesting to test a model by looking at practical applications rather than asking a series of questions that could be in the training data. You should consider making a series of videos in this format.
@johnkintree7633 ай бұрын
Excellent demo of Phi-3's RAG abilities. At the same time we seek a 3 billion parameters language model that runs well on a smartphone with at least 6 GB of RAM, we will also want a speech recognition model, and a dynamic graph neural network that can merge with a vector store to provide long-term memory.
@krisvq3 ай бұрын
Was thinking the same. We expect a lot from a small model.
@tvwithtiffani3 ай бұрын
I enjoyed your calm mellow speaking tone. Nice contrast to pretty much all of YT. Subscribed!
@tvwithtiffani3 ай бұрын
Question: Is there a local model that you would recommend for RAG? I've been building rag systems since gpt 3 (not 3.5) and I've yet to find a model that comes close to simply understand whats being asked at that given point in the conversation, extracting relevant info from stuffed context, and providing a response. I would even have gpt 3.0 (pre-chatgpt) quote the sentence from which it got its answer. My experience so far locally is that all of the moving parts outside of the local model have to be damn near 100% perfect to work correctly and even then the model will muck it up somehow every now and again, to the point its unreliable. Which models do you recommend for this specific use-case?
@engineerprompt3 ай бұрын
I personally like the zypher models if you are looking for smaller LLMs. For bigger local LLMs, llama-3 70B is good (in my use cases) and also CommmandR+.
@NoidoDevАй бұрын
19:33 - The model should only decide if something is a mathematical question, and then the script should decide that it has to use a tool.
@patrickscheich75323 ай бұрын
Nearly Perfect! Somehow my agent does not use tools for all questions, including the ones about meta, but the rest works :)
@outeast10523 ай бұрын
Bro, please, fix sound level. It's too quiet. I'm on 100 and can't hear anything, while all other videos are fine on 30.
@borisrusev94743 ай бұрын
I don't get the concept of multiple vector stores. How do they differ? Do they store different documents? Use different embedding models? Or maybe the chunking strategies are different?
@engineerprompt3 ай бұрын
In this case, each store will contain different docs. Imagine you have different knowledge bases for different departments and you want to retrieve info from the relevant department just based on the query
@jaysonp94263 ай бұрын
Couldn't you just add a different meta filter though? Is there a computational advantage to multiple vector stores?
@aa-xn5hc3 ай бұрын
These tests are really great! Please recommend what are the best llms for these purposes, at the time of making your tests.
@yotubecreators473 ай бұрын
Perfect content but the camera consumed me I am not able to focus I hope I can find similar content with normal static camera view like other videos
@trilogen3 ай бұрын
When running these what caliber of computing power are we talking? Any mid-high end laptop or mid-high end PC rig with good graphics card?
@engineerprompt3 ай бұрын
For this model, you will be able to run it on 6-8GB of vRAM. Potentially even with CPU.