I personally learn best through side projects and work/consulting projects - a just-in-time learning sort of approach, with hands-on and watching videos for topics as needed. After working through fine-tuning for a few projects, the concepts and terminology become way easier. The bigger picture, for real projects, will often looks something like this kzbin.info/www/bejne/j6WypZRmeZ1gaMU.

@@nodematic That makes a lot of sense. I'm currently just learning how to use models in Python in my own projects (though I'm still struggling quite a bit with them), but I hope this will get easier with time and I can start fine-tuning.

It sounds like your authentication isn't setup. Try `gcloud init` to set that up.

If it's a huge docx file, you could break it up and fine-tune on that. If you just need some sort of RAG/grounding, a no-code RAG solution like this would be best kzbin.info/www/bejne/fWTZqn5rj8yGoLM (works with docx files).

@@nodematic thanks, would data preparation be the same no matter the file type? just extract the text then convert it into key pairs?

For fine tuning, yes, you'll just need the extracted text to train on. Something like python-docx could do the extraction. Split up the extracted text into small enough samples that you don't run out of memory during the fine-tuning process (get the best GPU you can for this, like a Colab A100).

I would suggest Google Cloud Agent Builder. The service has developed since this video, but this will give you the idea kzbin.info/www/bejne/n5zGo5V9nZaDj6s. Basically, you add the PDFs to a Google Cloud Storage bucket, then create a no-code Chatbot Agent on top of that bucket.

Here's a video kzbin.info/www/bejne/fWTZqn5rj8yGoLM

That would work fine - no major concerns. Just keep in mind that the more structured the log data, the more useful and intuitive BigQuery-based analysis will be. Unstructured or semi-structured logs are going to be easier to analyze in Log Explorer.

Pro is required

I am getting the same error as well :(

you would first have to figure out the parsing logic to correctly extract the text and then put it in a summarizer, if all you want is a summary then there are many good models available on hugging face that you can use directly OR just get a gemini api key and use gemini for it, it should do a decent job at it.

@ thank you

@@igorcastilhos also if you want to use the PDFs as context to answer questions then you probably need to parse and then put them into a vector store so that they can be retrieved when needed, this is called RAG

@@abdulsami5843 I'm using Ollama with the Web UI tool. Inside it, I'm sending to the Knowledge collection some PDFs of resolutions of attorneys, so that they can ask about them whenever they want. The main reason to use Ollama (llama3.2) instead of OpenAI API is that it is free. But I'm having problems accessing the Web UI localhost:3000 from our server in my machine, it doesn't show the models installed in the server machine.

@@abdulsami5843 Also, the RAG feature doesn't have a very nice documentation. In my case, we have a distributed folder in microsoft windows (Like C:/) and inside that folder, the attorneys and advocates will send new PDFs through the website, and I wanted to use RAG for it, but it is very hard.

Thanks! Here's the notebook colab.research.google.com/drive/1vIrqH5uYDQwsJ4-OO3DErvuv4pBgVwk4?usp=sharing

At the time of video creation, the interface wasn't available, but we'll try to cover this in a future video.

You will always need both the base model and the adapter layers to actually run/use the model - it's just a question of if you want to merge and store everything "together". I haven't tried LM Studio, but upon a quick look, I would suggest saving/publishing your model as merged weights (should be simpler to pull into LM Studio). The "GGUF Conversion" portion of the fine-tuning notebook might actually be best for the export, based on LM Studio's website "LM Studio supports any GGUF Llama, Mistral, Phi, Gemma, StarCoder, etc model on Hugging Face". Hope that helps!

@@nodematic yes I've exported as gguf and used the merged option and it worked. although i faced with the model over fitting to a small database and that made it's behavior to go weird in some cases. but I'm working to expand my training data and maybe add different system prompts into the data as well... the question is, how to divide my data into training/test datasets and check the loss function for test dataset? does that colab notebook support it? or I'mma need to figure it out myself? ++ thanks for the answer. 💙

The loss is reported after each step in the training (you'll see this in the training cell in the notebook). A good approach is to see where the loss starts to "level off" (decrease significantly slower), and use the model at that point. You could also consider reducing the LoRA alpha value to put less emphasis on the adapter layers (and increase the emphasis on the base model). The expanded training set is a good idea, especially if you have less than ~200 examples. There isn't a traditional training/test split in fine-tuning like you would in other AI/ML problems - partly because responses are difficult to score quantitatively and with precision. Instead, people will do a post-training quality step of RLHF to integrate feedback on which fine-tuned model answers were good and which were bad, on tests. There are also some advanced methods to limit overfitting, but it's well beyond the scope of most small model fine tuning.

The demonstrated setup requires a load balancer - you'd have to DIY something if you're routing traffic straight to your VM public IP.

Thanks. We'll add that to our video ideas.

@@nodematic can't wait for that to come out! 💯 keep it up! 💪🏼

In my mind I'm trying to use about 10 review texts of my articles. And 1 expanded summary. I want the AI to write like me without AI plagiarism detection.