Building a Q&A Chatbot using GPT and embeddings

This livestream explains how to build Buster 🤖, a chatbot that answers questions related to the Hugging Face transformers library while citing its sources.
** LINKS **
Talk to Buster: huggingface.co/spaces/jerpint...
Github: github.com/jerpint/buster
Notebook: colab.research.google.com/dri...
In this tutorial we explain the 3 main ingredients necessary to make Buster work:
1) Collect data from the documentation
2) Build a document retrieval system and rank the most relevant sources
3) Generate text based on a user's question and answer the question

[@markgarrett2971]

So many questions and ideas! Thanks for this!

[@rvdjc]

Thank you for the amazing session!

[@WhatsAI]

Amazing tutorial, thanks Jeremy!

[@venividicredi4993]

Great info thanks. Love that buster started doing his own research (though appreciate why that might complicate integrity testing of the model)!

[@kishangovind8499]

Many many thanks and I wish you all the very best for your youtube chanel. Keep creating the video and help us learn. Could you next time show how to create a end-to-end deep learning model and deploy it

[@jerpint]

what kind of model would you want me to deploy?

[@cocoarecords]

fucking what i needed, thanks a ton. Subbed & liked

[@firasnacef001]

Thank you for the detailed process. Do you think the model can be finetuned to a language that chatgpt doesn't fully understand yet? (Let's say dialects of chinese or Arabic for instance, but written in latin characters). Does this method create a new vector space representation that will enable the model to understand this new language based on the input data? Or does it have to understand the input language first in order to generate the proper responses on which it will be trained?

[@jerpint]

there's a clear 2 step process in this solution: generating embeddings, and generating a response. To generate embeddings in a new language, the model has to have been trained on that language. You can definitely fine-tune models on other languages to do your retrieval. The generation part is trickier, because it requires GPT which can't be finetuned, but there are more and more clones popping up that allow you to finetune

[@LedZeppelinThe]

Is there a way to integrate this to Bubble?

[@ajithshenoy5566]

Hey Jeremy , did you figure out why the "Out of Scope" prompt wasn't working?

[@jerpint]

never figured out, hard to say with openAI closed models, but updating the prompt did help

[@dexter_chann]

does it has converstaion memory?

[@jerpint]

Not yet but hoping to add that feature soon

[@01juren]

I think that in the prompting, you need to be more specific like: " Question: .... and answer: ....... " At time: 55:47

[@MegaFristy]

Can you convert the model to ONXX format

[@jerpint]

The model under the hood is GPT, can’t be converted because we don’t have the weights :/

[@VaibhavShewale]

sed, to even play with it you gonna need money! i thought we would create our own chat bot or something :(

[@jerpint]

You can play with it for free on the huggingface space itself, but to deploy your own you have to pay OpenAI

[@markgarrett2971]

What would it cost to run a chatbot like this?

[@jerpint]

@@markgarrett2971 openAI charges by the token, and price per token varies by model. From experience,

[@Hydrosal]

HI Jeremy, I have a project for you, let me if you would be interested