Classifying customer messages with LLMs vs traditional ML
We’re classifying gigabytes of intel (SOCMINT / HUMINT) per second and found semantic folding or better in classification quality vs throughput than BERT / LLMs.<p>How it works — imagine you’re having these sentences:<p>“Acorn is a tree” and “acorn is an app”<p>You essentially keep record of all word to word relations internal to a sentence:<p>- acorn: is, a, an, app, tree
Etc.<p>Now you repeat this for a few gigabytes of text. You’ll end up with a huge map of “word connections”.<p>You now take the top X words that other words connect to (I.e. 16384). Then you create a vector of 16384 connections, where each word is encoded as 1,0,1,0,1,0,0,0, … (1 is the most connected to word, 0 the second, etc. 1 indicates “is connected” and 0 indicates “no such connection).<p>You’ll end up with a vector that has a lot of zeroes — you can now sparsify it (I.e. store only the positions of the ones).<p>You essentially have fingerprints now — what you can do now is to generate fingerprints of entire sentences, paragraphs and texts. Remove the fingerprints of the most common words like “is”, “in”, “a”, “the” etc. and you’ll have a “semantic fingerprint”. Now if you take a lot of example texts and generate fingerprints off it, you can end up with a very small amount of “indices” like maybe 10 numbers that are enough to very reliably identify texts of a specific topic.<p>Sorry, couldn’t be too specific as I’m on the go - if you’re interested drop me a mail.<p>We’re using this to categorize literally tens of gigabytes per second with 92% precision into more than 72 categories.