Train a Small Language Model for Disease Symptoms | Step-by-Step Tutorial

Dive into the world of Language Model as I guide you through the process of training a small language model using GPT-2! In this tutorial, we'll explore how to leverage the powerful distilgpt2 transformer to understand diseases and symptoms better.
📋 Tutorial Highlights:
Dataset Loading: Learn how to load a relevant dataset on diseases and symptoms from Hugging Face datasets.
Tokenization and Model Setup: Understand the crucial steps of tokenization using GPT-2's tokenizer and initializing the language model.
Training Loop: Walk through the training loop, exploring each epoch, monitoring training and validation losses, and ensuring your model is learning effectively.
Hyperparameter Tuning: Fine-tune your model by adjusting batch sizes, learning rates, and more.
Text Generation: Witness the power of your trained model by generating meaningful text based on input strings.
🤖 Why Train a domain specific Language Model like MedLLM?
Training a language model allows you to teach your model about the relationships between diseases and symptoms, enabling it to generate informative and context-aware responses.
🔔 Don't forget to like, comment, and subscribe for more exciting tutorials on Gen AI and machine learning! Your support keeps the channel thriving.
Join this channel to get access to perks:
/ @aianytime
📁 Download Code: github.com/AIAnytime/Training...
📚 Resources:
Hugging Face Model: huggingface.co/distilgpt2
Dataset Source: huggingface.co/datasets/Quyen...
#generativeai #llm #ai

[@Hellow_._]

subscribed just after watching one video. Thanks to creator. plz keep posting videos related to GenAI.

[@sohailhosseini2266]

Thanks for sharing!

[@AIAnytime]

Thanks for watching!

[@zeta_meow_meow]

saving, will watch later when i am good in ml and python

[@nunoalexandre6408]

Love it!!!!!!!!!!!!!!!!!!!!!!!

[@Mr.Tom.handle]

Need a video or instructions to integrating the model into mobile application

[@siriyakcr]

Much needed

[@VaibhavPatil-rx7pc]

Excellent information 🎉

[@AIAnytime]

Glad it was helpful!

[@namashaggarwal7430]

Awesome tutorial ❤ Can you please make a tutorial on how to fine tune a model, especially on textual and image data?

[@user-iu4id3eh1x]

Oh wow .... Thank you for this tutorial ❤

[@AIAnytime]

Glad you like it!

[@sriharsha5701]

@@AIAnytime i am having 35 columns if i ned to get all information based on ID how i would train please explain

[@alroygama6166]

I have a classificationstask where i have a para of text and it classifies labels of the para. Can i use this similar approach to tune it? Also will llama be a better choice?

[@adityashinde436]

my usecase is that the input will be some product keyword and output should be product category it belongs to (example - input: white shirt, output: apparel). for this purpose which model will be suitable, distilgpt2 is good or do you recommend some other models in text generation section or do you recommend me to check model from other section like roberta or distilbert?

[@ruksharalam173]

what extensions are you using for code autocomplete in colab?

[@sriharsha5701]

i am having 35 colum,s if i ned to get a;; information based on ID how i would train please explain

[@subramanianchenniappan4059]

Please tell if you have any training on how to train and deploy a model given a dataset . I am an experienced developer want to learn machine learning and AI and related stuff😊😊

[@subashpaudelhimself964]

provide a code to push this model to hugging face too sir.

[@jorgerios4091]

The steps presented for training the model are incredibly valuable, thanks for your guidance. Would there be any notable differences in the results if we were to utilize a LaMini model instead of GPT2?

[@AIAnytime]

In my experiments, T5 was better .... I am working on that to create a video shortly for a similar use case.

[@ARkhan-xw8ud]

is there any metrics to check the performance of the LLMs @@AIAnytime

[@AIAnytime]

Yes look at my Evaluation of LLMs and RAGs video. It's detailed.

[@ruksharalam173]

What are the differences between encode and encode_plus?

[@user-qi4jw1lf9i]

WOW IT IS JUST A AWESOME I HAVE 36000 DOC AND WAS USING LAMINI ' YOUR VEDIO WITHOUT INTERNET BUT IT IS HOW CAN I BUILD MODEL USING THAT DOCS SO FETCHING TIME WILL BE LOW

[@dragneel4451]

Can I train this model on movies dataset with columns Name, Synopsis, Genres and ask the model to recommendation similar kinds of movies?

[@AIAnytime]

Absolutely. You must do it. Try T5 model instead distilGPT

[@jorgerios4091]

Buddies, it runs on colab but when I try it in my Windows VS code environment I get the following error: RuntimeError: PyTorch is not linked with support for mps devices. Anybody else got the same and how did you fix it?

[@jorgerios4091]

Got it now, Gemini told me mps is just for Apple 🤡

[@akash_a_desai]

Convert it to ggml format so we can use it in cpu

[@AIAnytime]

Won't recommend to convert this into GGUF as the model is already very small (around 380MB). Compressing it more will further degrade the performance. But if the data quality is better and other LMs like t5 and lamini are used then we can try GGUF. The current one smoothly runs on CPU tho.

[@nothing_is_real_0000]

@@AIAnytime Agree. Any model within 1 GB is very much within "small" model range

[@Sergatx]

Why didnt you do this with GPT 3.5 instead ?

[@pjoshi_15]

If you don't want to expose your data to open AI

[@AIAnytime]

1. Gpt 3.5 in a closed source. You need to pay. 2. Data protection and privacy. 3. Inference token costs.

[@asiddiqi123]

​@@AIAnytime So you don't pay Hugging face?

[@sumitawasthi7068]

Why did Manchester United bring a map to the game against West Ham?

[@CooperDaniel87]

MAN U 💔

[@AIAnytime]

❤️😜

[@mingyukang6592]

Sorry, I have a beginner question. 1. What is the difference between using data in a data frame and using it by creating a model and loading it like now? 2. Is the 'SmallMedLM.pt'' model created in the example simply a vector database of "QuyenAnhDE/Diseases_Symptoms" data? 3. Or do you mean that ‘SmallMedLM.pt’ is a GPT2-based LLM fine-tuned with symptoms matching the disease? 4. So, does this mean that the trained 'SmallMedLM.pt' can be used when creating a chatbot related to a specific disease or symptom?