Finetuning Open-Source LLMs

This video offers a quick dive into the world of finetuning Large Language Models (LLMs). This video covers
- common usage scenarios for pretrained LLMs
- parameter-efficient finetuning
- a hands-on guide to using the 'lit-GPT' open-source repository for LLM finetuning
#FineTuning #LargeLanguageModels #LLMs #OpenAI #DeepLearning
Useful links to resources discussed in this video:
Code for the LLM classifier: github.com/rasbt/LLM-finetuni...
Lit-GPT repository: github.com/Lightning-AI/lit-gpt
NeurIPS LLM efficiency challenge: llm-efficiency-challenge.gith...
My latest articles on LLM research: magazine.sebastianraschka.com/

[@Dom-zy1qy]

Very much appreciate this video, fine-tuning seemed like a somewhat amorphous concept to me for sometime, but the diagrams you showed really made it easier to understand how people finetune.

[@SebastianRaschka]

Thanks so much, glad these diagram were helpful and helped clarifying!

[@captinbo1]

Thanks! Great overview

[@mysticaltech]

Awesome, thank you!

[@prakhargurawa]

Thank you :)

[@nadranaj]

Thanks

[@Mayur7Garg]

One of the approaches I have experimented with, which is both manual labor, time and compute expensive but more reliable, is as follows: - Use a LLM to query for outputs. Use RAG and prompt engineering to get the best possible results. - Generate chat logs for each query. The log should include everything - the prompt, the retrieved info if any and the model output. Any special symbol such as to denote the system prompt or anything else should also be left in. This is because LLMs are text generation models with no concept of chat. - Manually update the model outputs to better reflect the expected output. This is a data creation task. - Fine tune a copy of the same LLM using PEFT using the updated chat logs. This can also be done iteratively as long the chat logs are generated initially by a model which hasn't been fine-tuned yet. Like a sort of A/B experiment. Some use cases are served the original model that generates the data for fine-tuning while the other are served the fine-tune model whose outputs are not used for any further fine-tuning. Expensive but over time, your model would work better for realistic inputs.

[@lalmuansangachhakchhuak4927]

Cool

[@mohammadkad]

Amazing, Thanks

[@zjffdu]

Thanks for the video, very helpful for me to understand different kinds of finetunning. BTW, what kind of finetunnig is huggingface belong to?

[@SebastianRaschka]

Glad that it was helpful! HF itself has different tools for finetuning. Similarly, the LitGPT library I help developing supports full finetuning, LoRA, QLoRA, etc.

[@muhammadanas7698]

Time saw you here on YT! Hope you remember me.!