Revolutionizing Search: How to Combine Semantic Search with GPT-3 Q&A | Full Code Tutorial

[@FutureSmartAI]

📌 Hey everyone! Enjoying these NLP tutorials? Check out my other project, AI Demos, for quick 1-2 min AI tool demos! 🤖🚀 🔗 KZbin: www.youtube.com/@aidemos.futuresmart We aim to educate and inform you about AI's incredible possibilities. Don't miss our AI Demos KZbin channel and website for amazing demos! 🌐 AI Demos Website: www.aidemos.com/ Subscribe to AI Demos and explore the future of AI with us!

[@sairam332]

This video is a power house of practical knowledge. Really useful. Thank you.

[@FutureSmartAI]

Glad it was helpful!

[@desuharshith4276]

thank you so much bro.. i have project ont this...iam using your video bro..

[@ArunKumar-bp5lo]

i was looking for video on semantic search 😅😅 thank you .perfect timing

[@FutureSmartAI]

You're welcome 😊. Check Description Added Code

[@sadoubarry5022]

Thanks for sharing. Une chaîne KZbin très intéressante pour apprendre !!

[@advaitbhake5152]

This is great work Pradip , thanks for sharing your knowledge. God Bless ! You will soon have million subscribers!

[@FutureSmartAI]

So nice of you! Did you check my recent video on same topic which covers GPT-4, Langchain and Pinecone

[@advaitbhake5152]

@@FutureSmartAI sure , I will watch ! By Any chance you can share the pinecone key ? We are trying to create a prototype and pinecone has a wait list currently. I can understand if not.

[@advaitbhake5152]

@@FutureSmartAI also do you recommend vector databases that can be on premises?

[@FutureSmartAI]

@@advaitbhake5152 No I dont think Pinecone has waitlist. Also, pinecone is costly, you can use an opensource vector db called Chroma www.trychroma.com/

[@advaitbhake5152]

Do you have any tutorial on chroma embeddings?

[@johnny1003]

Subscribed. Thank you Pradip!

[@FutureSmartAI]

Awesome, thank you!

[@playboy71322]

Nice video, how hard would it be to select the database if you have multiple pinecone databases with different information that you want to include in your overall corpus of data to ask questions against.

[@FutureSmartAI]

Not much, you can have different indexed with different info and you can compare most similar items from all indexes

[@playboy71322]

@@FutureSmartAI I haven't seen that video anywhere on YT. I have been searching today.

[@duetplay4551]

. ... One further question: for user friendly, how can we highlight the "chuck" that was found against the original context, like what google did? Would you please give me some hints? thx

[@FutureSmartAI]

We are already showing chunks that we found relevant

[@duetplay4551]

@@FutureSmartAI yes, I found the way to highlight the chunk within the context. Thanks 😄

[@RajeshGupta-gx3yz]

Hi @duet play, would you mind sharing how you highlighted the context within the chunk? 😄 would be interesting to see!

[@duetplay4551]

@@RajeshGupta-gx3yz if chunk in contents: print (contents.replace(chunk, f'\033[33;04m{chunk}\003[m'))...It works for small LM, not for GPT3 because it re-write the answer.( My individual experience though😌)

[@kyoungd]

Great stuff. Thanks .

[@FutureSmartAI]

Glad you enjoyed it

[@venkatesanr9455]

Great job and interesting/insightful video. I have one doubt how to handle concurrent restful async api calls in python if you know some inputs on this. Kindly provide.

[@FutureSmartAI]

We have done some work on it recently we will create video.

[@RajeshGupta-gx3yz]

Hi @Pradip Nichite, great content! Thank you for sharing your knowledge! Just wondering how you’d handle the inaccuracies from GPT-3 while summarizing content? For example, not sure if the Amitabh Bachchan summary was exactly right considering it was a document on Sachin Tendulkar. Isn’t it a case of hallucination (or accessing it’s pretrained knowledge)?

[@FutureSmartAI]

Hi Rajesh we could put some threshold on semantic score. If it matches score and gives answer based on factual information present in our own docs then should be fine.

[@RajeshGupta-gx3yz]

That makes sense.. Thank you once again! Excellent content! (:

[@nisarshah1355]

Hi Pradip, Thanks for sharing this, very useful for building applications in this domain. Also, Is code shown in this video available for public access. Also could you please mention those tutorials/videos which you have shown in video, required to get better understanding of the concept.

[@FutureSmartAI]

Updated all details in description.

[@nisarshah1355]

@@FutureSmartAI Thanks Pradip.

[@pa11001]

Thank you for sharing. Is the openai.api_key free?

[@FutureSmartAI]

No, When you register you will get some free credits

[@shrutinathavani]

hello sir can i do the update database with pdf and not URL? Thank You

[@FutureSmartAI]

Yes. check my other videos

[@shrutinathavani]

@@FutureSmartAI i looked at some of your videos but did not get any idea could you provide a better way?

[@FutureSmartAI]

@@shrutinathavani In this video I have processed text file and pdf file to update pienceone db kzbin.info/www/bejne/mYekYoWGqKyEmdk

[@FutureSmartAI]

@@shrutinathavani Steps remain same as url, only you will use pdf processing library like pypdf2 to extract text and then insert in db

[@shrutinathavani]

@@FutureSmartAI ok thankyou helped alot

[@RamaChetan]

you should have used text embedding ada 002

[@FutureSmartAI]

I have used it in others videos

[@APARAJEET_SAHOO]

Rather using openAi which pre trained model would help us more, openAi is not meant for enterprise,so don't want to use

[@FutureSmartAI]

yes there opne source alternatives available and they are not as good as open ai. "openAi is not meant for enterprise?" Can you elloborate on this?

[@APARAJEET_SAHOO]

@@FutureSmartAIis our data not exposed ,when we connect to openAI ?

[@FutureSmartAI]

@@APARAJEET_SAHOO Though compnaies dont like that. Many compines does use clould apis and pass their data including ocr apis, llms api. Since Quality difference is so high compnaies ready to pay open ai and pass their data to stay relevant in this world. Either they can wait till open source llms as good as open ai, Build their own LLMs or simply become obsolet.

[@FutureSmartAI]

Here Leaderboard that shows open source LLM performance. You can see which is best open LLM. huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

[@APARAJEET_SAHOO]

@@FutureSmartAI what's ur thoughts on pre trained models which r build for specific context like text2sql ? I feel and see those small size models are doing great

[@wilfredomartel7781]

Great ! But gpt 3 is paid.

[@FutureSmartAI]

Yes but its worth paying when you compare it with other alternative. Also its getting cheap by every release.