17 Year Old Builds a Better Search Engine
Armchair analysis of his algorithm after watching his TED talk: a version of LSA that uses PageRank instead of a straight SVD to calculate rankings.<p>LSA[1] has been around since the 80s and is used in many applications from GRE testing to Apple's junk mail filtering[2]. It's used a lot since the patent expired, it's relatively good and can be computed quickly. Of course, a lot of text-retrieval research has happened in the past few decades, one of my favorites being LDA[3] which relies on a much more sound statistical basis than finding lower-dimensional representations of term-document vectors. Unfortunately LDA's model is not directly computable and answers must be determined via Monte-Carlo methods.<p>As for 'indepdendence,' his terminology gets a little confused here. At first I thought he was talking about the 'bag-of-words' assumption that most large-scale language models have. These effectively ignore grammar (other than stemming) in order to efficiently determine the 'gist' of a document without its intricacies. However, his videos imply he is talking about word-sense disambiguation[4], which is certainly known about and was the crux of LSA in the first place. If he <i>is</i> talking about lifting the bag-of-words assumption, there has been some interesting work going on, such as [5] (disclaimer: I am a coauthor on that paper).<p>If you're interested in this stuff, I highly recommend trying out the LSA demo server at [6] (it can get swamped sometimes so don't kill it) and David Blei's LDA implementation at [7]. The LDA-C inputs and parameters are a little obtuse when you first look at it, and I don't have my notes on how to use it at the moment but if you play around with it it should make sense.<p>This kid is crazy smart, and I hope he gets exposed to a lot of really cool research since he can obviously pull off a lot at a young age. Best of luck to him.<p>[1] <a href="http://en.wikipedia.org/wiki/Latent_semantic_analysis" rel="nofollow">http://en.wikipedia.org/wiki/Latent_semantic_analysis</a><p>[2] <a href="http://developer.apple.com/library/mac/#samplecode/LSMSmartCategorizer/Introduction/Intro.html" rel="nofollow">http://developer.apple.com/library/mac/#samplecode/LSMSmartC...</a><p>[3] <a href="http://en.wikipedia.org/wiki/Latent_Dirichlet_allocation" rel="nofollow">http://en.wikipedia.org/wiki/Latent_Dirichlet_allocation</a><p>[4] <a href="http://en.wikipedia.org/wiki/Word-sense_disambiguation" rel="nofollow">http://en.wikipedia.org/wiki/Word-sense_disambiguation</a><p>[5] <a href="http://aclweb.org/anthology-new/D/D12/D12-1020.pdf" rel="nofollow">http://aclweb.org/anthology-new/D/D12/D12-1020.pdf</a><p>[6] <a href="http://lsa.colorado.edu/" rel="nofollow">http://lsa.colorado.edu/</a><p>[7] <a href="http://www.cs.princeton.edu/~blei/lda-c/" rel="nofollow">http://www.cs.princeton.edu/~blei/lda-c/</a>