How LSH Random Projection works in search (+Python)

[@FirstNameLastName-gh9iw]

Thanks for the explanation. I’m researching what I should use in my program and without explanations I’m just grasping at straws. I don’t have too much experience with these type of things so I’m trying to expand my horizons to see if there are things I am missing

[@AakashOnKeys]

James, your videos deserve better reach! Neatly explained and very useful!

[@HugoOlsson01]

Thank you! Really good explanation.

[@gorgolyt]

Also, does FAISS LSH really use hyperplanes through the origin? If you look at the diagram at 18:18 you can see that's really not ideal, as you split the space into large cones, and anything within that cone -- close to the origin or far away -- is considered "near". This will probably work well if the vectors are unit vectors, but otherwise not. Other methods like Annoy use the data to choose the hyperplanes, taking the hyperplanes between two random points. These don't go through the origin and can partition the space into bounded cells, which seems a lot more sensible.

[@gorgolyt]

Does FAISS LSH really hash the binary vectors, or does it just use their binary values, which would be simpler? For instance for nbits = 4. You don't need to hash [0, 0, 0, 0], [0, 0, 0, 1], etc., you can just directly use pointers corresponding to 0000, 0001, 0010, 0011, etc.

[@Neira1991]

gracias James

[@Leela_Karthik]

well explained by the way, what ide is it? looks nice

[@jamesbriggs]

VS Code with Jupyter extension :)

[@Leela_Karthik]

@@jamesbriggs theme?

[@warrenhenning8064]

It should be possible to simplify the code in the for loop: ``` import collections buckets = collections.defaultdict(list) for i, vector in enumerate(vectors): hash_str = ''.join(vector.astype(str)) buckets[hash_str].append(i) ```