A Small Language Model (SLM) using Python

Let me guess, you are familiar with Large Language Models or LLMs, or at least you have heard of them. An example is GPT, which is the backbone of ChatGPT. A part of the current AI revolution is coming from large language models. Speaking of LLMs, a critical question is, what is a small language model, or SLM?
A small language model generally refers to a language model with a lot fewer parameters, simpler architecture, or limited training data compared to LLMs. Small language models are faster to train and use less computational resources. SLMs typically have less capability to understand and generate complex language.
Can we create a ChatBot using a small language model? Yes, that is exactly what we are going to do today, we will write a ChatModel using an SLM.
We will be using a very very small language model, a language model so small that it might not even be considered a small language model. We will be using a bigram model to create a ChatBot. The ChatBot we will create will require one word as a prompt, and the bot will generate as many words as you ask for. The quality of the generated text will not be that awesome, but it is fine because today’s entire practice is for clarity of how a basic small language model can be used to build a basic chatbot.
I will use no neural networks, and no complex coding. I will use old-fashioned NLTK for creating the bigrams.
Here is the code for the Chatbot using the SLM: colab.research.google.com/dri...
Here is the code to create the data folder: colab.research.google.com/dri...
Dr. Shahriar Hossain
computing4all.com

[@ashraf_isb]

kudos to you, you did this 5 months back where now people are taking about slms! good job and thanks for this!

[@C4A]

Thank you for the kind words!

[@kunalsoni7681]

Amazing After All :) small pack with a big blast

[@C4A]

Glad to hear that you liked it. Thank you for watching!

[@RiteshKumar-zy6zt]

sir, can you suggest any small language model that could generate good structured response and text summarization.

[@deeplearner-hinglish]

No, this is only for 2-3 words answer responses in chatbot, This time is gone. Building a chatbot using Bigram SLM is good. But we cannot frame a whole sentence by this. In my opinion this is a failure.

[@C4A]

Absolutely, this video is for educational purposes, just to demonstrate how the simplest idea works. The code is not intended for any functional chatbot. Thank you for watching.