Practical and Optimal LSH for Angular Distance
We show the existence of a Locality-Sensitive Hashing (LSH) family for the angular distance that yields an approximate Near Neighbor Search algorithm with the asymptotically optimal running time exponent. Unlike earlier algorithms with this property (e.g., Spherical LSH [1, 2]), our algorithm is also practical, improving upon the well-studied hyperplane LSH [3] in practice. We also introduce a multiprobe version of this algorithm and conduct an experimental evaluation on real and synthetic data sets. We complement the above positive results with a fine-grained lower bound for the quality of any LSH family for angular distance. Our lower bound implies that the above LSH family exhibits a trade-off between evaluation time and quality that is close to optimal for a natural class of LSH functions.
The authors propose a novel LSH family for angular distance which (a) matches the theoretical guarantees of Spherical LSH (i.e., an asymptotically optimal runtime exponent) while at the same time (unlike Spherical LSH) being practical in that they outperform Hyperplane LSH for the same task by up to an order of magnitude.This result is achieved by the combination of techniques where (a) a data point is first hashed using a random rotation and then the after-image matched to the closest point y from {+/-e_i}, where e_i is the ith standard basis vector of \R^d, (b) the random rotation is performed as a sequence of 3 pseudo-random rotations (reducingthe asymototic overhead from O(d^2) to O(d log d), (c) feature hashing to reduce the effective dimensionality d, and (d) using a variant of multi-probe LSH for retrieval which generates additional probing locations.The proposed work is novel, original and has the rare property that it combines theoretical optimality with a practical, realistic (C++ instead of MATLAB) implementation.