Fine Tuning a Model in Gemini and Vertex AI | Steps to make a LLM

We walkthrough how how to fine tune a Gemini model in Vertex AI, that uses your input data to train the model. The video covers off where to tune, steps to follow on how to tune and create a JSONL file sample. We then show you where to find it and test it.
For videos on how to get access to Vertex AI: • Google Vertex AI
Links: cloud.google.com
Google AI Studio: ai.google.dev/
For a Full course on AI with Google Vertex and how to build your AI applications support me here: www.udemy.com/course/ai-on-go...
If you want a sample file let me know.
Contents:
00:00 Introduction
00:37 Vertex AI Language Models
01:01 Tune a Model steps
01:59 JSONL File overview
03:26 Create a JSONL file
05:21 Sample JSONL file - tips
06:21 Tuning Pipeline
08:30 Use the new Model
10:11 Model name in Google Cloud
#googleai #vertexai #geminiai

[@uncle_slashes]

Based on the information in Google's doc - "The PaLM 2 for Text (text-bison, text-unicorn) foundation models are optimized for a variety of natural language tasks such as sentiment analysis, entity extraction, and content creation. The types of content that the PaLM 2 for Text models can create include document summaries, answers to questions, and labels that classify content." This video is not demonstrating Gemini.

[@cnota69]

thank you so much for the instruction

[@LevelUp_Plus]

Thank you for watching, glad it was useful.

[@mr.daniish]

Loved it!

[@LevelUp_Plus]

Thank you, and glad you enjoyed it. I had fun making the video.

[@outcast2313]

great video! Thanks for this. Seems like its a lot simpler than having to understand how the tokenizer and architecture works and try to get your training dataset in that format. That being said, once tuned can the fine tuned model weights be downloaded to your local machine

[@jonzh4720]

Thanks. Can make video for evaluating finetune results such as accuracy in vertex?

[@LevelUp_Plus]

Hi thanks for watching, I will have a look into this. thanks for the suggestion.

[@noureddinhamouda3438]

thank you very much for this video, can you record another video for the chat-bison, I'm always getting an error when the training comes to the dataset-encoder.

[@LevelUp_Plus]

Hi, the encoder issue is due to the training dataset you are using, it needs to be formatted perfectly. below is a sample of one line items I used in my JSONL file. {"input_text": "question: What is Tyrone's view on the future of AI in insurance? context: AI is possibly the most exciting technology to ever have been created in my view, a solution that allows access to model and information at unprecedented rates","output_text": "The Gen AI future is looking awesome, so excited and can't wait to help more business adopt it"}

[@pradachan]

can you turn off the browser when you just uploaded jsonl file to get trained, would it get trained or i have to keep the pc on>?

[@LevelUp_Plus]

Hi thanks for watching, and yes you can close the browser and your PC, the the engine will run in the background and alert you once done.

[@John-jt3dn]

Hello.. Thank you for the video.. I have a question.. for inputs i have examples questions and 5 other input parameters and or course and output. How can i integrated there parameters into the trainin data set? Which format should i use? Thanks.

[@LevelUp_Plus]

Hi thanks for watching, I would check the documentation as a first port of call. There may be some experimentation here. So keep the number of samples low to test it first.

[@John-jt3dn]

It took vertex AI to fine tune the data 2 hours 12 minutes on my side just for 10 examples while i was testing. Is there a way to speed this up? My original data has 14467 examples. It would take ages....

[@LevelUp_Plus]

Hi, Thanks for watching and appreciate the comment. In Vertex AI, I have seen a way to allocate a more powerful server/TPUS's to the environment . Alternatively you may try the Google colab version which allows you to select the processing power you would like. I have only very briefly experimented with runtimes and different CPU configurations.

[@Ristontestaus-on8uq]

Hi there, thanks a lot for this. Trying to test somethign with it, but I get the "Failed to create pipeline job. Error: Permission 'aiplatform.metadataStores.get' denied on resource" error. I think it has to do with the service accounts and permissions. For me it only shows the compute engine default service account. I've gone to create service accounts but over there the instructions I've seen ask me to download a key, and that's no longer allowed. Could you please nudge me in the right direction on how to get the service account setup properly so that I could select it here? Thanks!

[@LevelUp_Plus]

Hi, if you haven't already you might need to go through the Google Cloud Setup steps to configure your environment, so click the Google Cloud Setup, and go through the wizard. The user access and service accounts are provisioned in there.

[@KadirOrtac-xj9ww]

i've got an error like this Failed to process dataset for gemini-1.0-pro-002: Dataset validation failed: {"missing_messages_list": while tuning model. What am i supposed to do?

[@LevelUp_Plus]

Hi thanks for watching, I found the biggest reason for the failures was the JSONL file. Check the format of the file and ensure the format and end lines etc are correct.

[@MakeChangeNow]

what is the use case for making one? and how is it different than making an agent?

[@LevelUp_Plus]

Hi Thanks for watching, the use case for refining a LLM is to provide it with context for your data/environment. For instance if your marketing material is based on a certain style or PR questions and answers follow specific format, or if you want the model to behave like your organisation. Or if you are a specialised insurance/finance with use cases that you have as knowledge banks then using your business IP you can train the LLM further to be more accurate.

[@ayushsinghal28]

how can we fine tune the multimodal Gemini AI pro

[@LevelUp_Plus]

Hi, the options are being made available in different countries at different stages. So depending on the country you are in, will be when it is available.

[@DeepakPoojary]

dude the code u copied at 3:20 and pasted at 3:50 are two different codes , can u please give this code

[@LevelUp_Plus]

Hi Thanks for letting me know - here is the JSON. below is a sample of one line items I used in my JSONL file. {"input_text": "question: What is Tyrone's view on the future of AI in insurance? context: AI is possibly the most exciting technology to ever have been created in my view, a solution that allows access to model and information at unprecedented rates","output_text": "The Gen AI future is looking awesome, so excited and can't wait to help more business adopt it"}

[@KadirOrtac-xj9ww]

can i use my tuned model in my web application for free?

[@LevelUp_Plus]

Hi thanks for watching, the costs for the models are in the vertexAI documentation, and will be token based.

[@berrabeser7155]

⁠​⁠@@LevelUp_Plus can we use another environment with the same code that we get from the vertex?