An intro to rerankers: A uniform API for reranking models

[@romainjouhameau2764]

Very cool lib indeed ! Thanks for all your videos Mark, always something to learn from your videos

[@learndatawithmark]

Awesome - glad they're helpful :)

[@willTryAgainTmrw]

Great! Also loved your parquet series when I watched it probably a year back. Subbed.

[@learndatawithmark]

Good to have you back and thanks for the kind words!

[@Kk-ed1gr]

absolutely useful tip

[@aryamasingh3413]

I didn't understand how you performed document/paragraph chunking? Is each line a "chunk"? Also, when you are creating the embeddings of each chunk, where are you storing them? In a vector database?

[@learndatawithmark]

Yeh each line is a chunk. In that dataset (extracted from a BBC news article) they had one sentence paragraphs, so I just made each sentence a chunk.

[@aryamasingh3413]

@@learndatawithmark Thanks!