Fine-tune ChatGPT? Buy Embeddings /OpenAI? What are Embeddings? My own ChatGPT? | Visual Q+A

[@Davipar]

Amazing content as always! best channel out there about the topic!! Thank you!!

[@code4AI]

Wow, thank you!

[@architectsmusicgroup]

You can say that again .

[@li-pingho1441]

Your content is so amazing. thanks a lot!!!!!

[@code4AI]

You're very welcome!

[@creativeuser9086]

Is the embedding model a completely separate model or is it a sub part of the GPT architecture (for example, the embeddings could be the initial part of the transformer architecture which says ‘embed’). In other words, did openAI have to train the embedding model separately or was it trained along with the GPT models ?

[@creativeuser9086]

Can you talk about open source embedding models. I find it hard to just list all of them and judge on their capabilities.

[@code4AI]

Sure. I have a specific video on comparing the performance of commercial vector embeddings: kzbin.info/www/bejne/mnebhoOgn8iMm5I

[@Ruramai-Technologies]

@code4AI thanks for the great content !i have been binge watching for days! Question so for this smaller and more specific neural search model, one can fine-tune or prefix-fine-tune a smaller model like FLAN-t5-small? or will it be wiser just to make a system that just goes to lets say Wikipedia or google search and return a more relevant answer?

[@code4AI]

The BERT encoder stack architecture is especially great to pre-train and fine-tune. You can find some videos on my channel about it, either in PyTorch or Tensorflow2/KERAS. But remember: T5 are full stack transformer models, you need a lot of compute power to fine-tune them.

[@i_accept_all_cookies]

Great analogy, well done!

[@code4AI]

Appreciate your positive feedback! Thanks.

[@danson3038]

which embedding model would you recommend for onpremise (not openai cloud)?

[@code4AI]

Microsoft's CEO announced, that Microsoft will make ChatGPT available to corporations, trained on their particular needs and tasks. Which is a logic move, since the full fledged ChatGPT is not cheap to run in the cloud and this general ChatGPT sys for everybody might not have detailed enough information for specific industrial sectors or industries. So a move to a highly specialized system is a logic path forward. So you will have a biomedical GPT (like BioGPT from Microsoft) and further specializations. It al depends, if you want to deploy a system that is trained on generative tasks (like GPT, the decoder stack of a transformer), on language understanding tasks (like BERT or for more complex systems Sentence Transformers (SBERT, the encoder stack of a transformer)) or for a combination of both (like the complete transformer architecture) like eg T5. Your task define the optimal system. ..... Example ..... If I search my scientific literature for thematic correlations in diverse scientific topics, I use SBERT, since my focus is on detailed scientific information, I choose a system with high performance and trained on understanding human language to the highest degree, without (!) paying 5000 - 12000 US$ for a cloud supercomputer usage per year.

[@danson3038]

@@code4AI wow. what a detailed answer!!!!. its a video script indeed. thanks again for you time and experience. will start with bert like systems and then t5 depending on use case. always opensource!