Supercharge eCommerce Search: OpenAI's CLIP, BM25, and Python

[@yamani3882]

A demo of what we are about to learn in the beginning of the video would greatly help an infant such as myself in this field.

[@jamesbriggs]

good to know, will implement this in future videos, thanks!

[@frackinfamous6126]

He's got a ton of vids. If you go back you can find the setups multiple times I'm sure. I think it's kind of refreshing that you jump right in. I know that sounds like i'm saying don't help new people but you'll soon see there are 17,000,000 videos showing you how to get an OpenAI key and what the basics are.

[@iknowsolittle]

This channel is shockingly good for its subscriber count. Lucky I found you. Thanks!

[@jamesbriggs]

glad you're enjoying it, thanks!

[@frackinfamous6126]

Just goes to show subs have nothing to do with quality of the content. Subs is a vanity number and James will definitely get them. Honestly most coding KZbinrs that go for subs just rehash the same stuff over again with a different "project" on top. Usually get stuck 20 minutes in because they don't know how properly set up their environments and then edit out 3 seconds that they shouldn't have. Then you rage quit. You're out a laptop, a girlfriend and still can't code. At least that was me coming up lol.

[@gowthamkrish773]

I'm using s1 pod and trying to create an hybrid index with 10k vectors. Will there any pricing difference between using a dense vector index alone and using a dense+sparse vector index from pinecone side?

[@JasonMelanconEsq]

This video is great! Instead of running on Colab, could you make a video that shows an up and down connection from an html front end to the Pinecone database, specifically uploading a PDF, vectoring it, querying, and displaying the results back through html? I also emailed you for some consulting work on a project. Thanks for the videos!

[@adamswang]

very nice, the sparse and dense vector mix can apply to many sceanrios.

[@Cropinky]

nice explanaysh bro

[@chrismaley6676]

This demo is fascinating. I would love to learn what technology to add to extend the demo, to maintain context between queries.

[@jamesbriggs]

will be covering this sort of conversational AI stuff soon (next week I hope)

[@chrismaley6676]

@@jamesbriggs I'm going to try integrating chatGPT and experiment.

[@JuanLopez-oc9yv]

Amazing content as always. I was wondering, is it recommended to use embeddings such as the ones form Openai or cohere instead of BM25?

[@jamesbriggs]

use both, here I use the CLIP model for dense embeddings, this is equivalent to using OpenAI's ada-002 (with the exception that this would be for the text, and not the image) - the reason we add BM25 here is because these dense embedding models (CLIP, ada-002, etc) are good but struggle for exact matching of keywords, something that BM25 is actually pretty good at That being said, SPLADE is like the "next step" from BM25, I'd recommend you look into that too - I have a walkthrough using this coming early next week

[@JuanLopez-oc9yv]

​@@jamesbriggs Thank you!

[@hoangphanhuy1992]

@@jamesbriggs I am still struggle to understand why we need to be good at exatch matching of keywords in this scenerio, I mean the real benefit of exatch match keywords over CLIP. Thank you so much I dont know whether u understand or not but hopefully

[@RichardHamnett]

Keep up the fantastic content mate.

[@jamesbriggs]

will do

[@timvw01]

Well explained, interesting!

[@jamesbriggs]

thanks!

[@kristiansopkovic697]

Good job ! :)

[@jamesbriggs]

Thanks!

[@JohnKing93]

Is there a reason why you didn't use CLIP to generate both image and text embeddings?

[@tommyle187]

Yes why did you not just use clip for both? @jamesbriggs

[@hemanshupan]

Hello James, great content. I have 1 query. How do we handle the query "show me blue jeans under $50", this "under $50" value while building a search engine. If you can guide me, would much appreciate it, thank you.

[@nicholaschu203]

From Pinecone filter property we could use {"maxPrice": {"$lt":[50]}} But is it how data should be structured? i.e. Having the price within the row/entry. I'm curious to see what others do in real world because each of these Pinecone queries with embedding searches cost tokens/money.

[@DAWEAP1]

Great stuff!

[@atomhero2830]

Hi thanks for sharing the video it is really useful. For this type of usage, other the Pinecone are there any other vector DB that run offline on local machine?

[@jamesbriggs]

faiss is a good option if you're wanting to do something (not too big) offline

[@shahzainhaider2801]

Discord link?

[@markcuello5]

HELP