Table Small Collection, Stanford NER comparison of results for location between standard training set, and custom training set. CUSTOM STANFORD NER STANFORD NER lanza 217 lanza 339 adam 197 obama 264 obama 33 adam 246 ryan 16 sandy 140 nancy 9 hook 129 vance 4 john 102 paul 4 edwards 97 summers 3 kasich 84 puja 3 paul 82 3 jennifer 69 nather 3 christina 67 lanzas 3 scarlett 63 Table Small Collection, Stanford NER comparison of results for location between standard training set, and custom training set. As visible from the results, this approach gave us a much better representation of the content of the collection when compared to using just word frequencies or POS tagging. The results obtained from our named entity recognition improved upon previous methods in providing unique indicative information for our collections. 2 topic 1 WORDS PROB WORDS PROB school 0.0175 days 0.0232 shooting 0.0085 ago 0.0159 elementary 0.0073 point2 0.0094 hook 0.0071 permalinksavecontextfull 0.0088 news 0.0068 2.0000 0.0076 sandy 0.0067 ago 0.0099 newtown 0.0067 you 0.0072 conneconnecticut 0.0057 I 0.0070 2012 0.0050 all 0.0056 from 0.0043 points1 0.0054 december 0.0041 reddit 0.0052 Table Small Collection, best topics Latent Dirichlet Allocation A topic model is a statistical model that aids in the discovering of abstract occurring in a “topics” collection . If a document is about a particular topic, we would expect that certain words would appear in the document more or less frequently. Latent Dirichlet Allocation (LDA) is a way of automatically
Table Small Collection, Stanford NER comparison of results for location between standard training set, and custom training set. CUSTOM STANFORD NER STANFORD NER lanza 217 lanza 339 adam 197 obama 264 obama 33 adam 246 ryan 16 sandy 140 nancy 9 hook 129 vance 4 john 102 paul 4 edwards 97 summers 3 kasich 84 puja 3 paul 82 3 jennifer 69 nather 3 christina 67 lanzas 3 scarlett 63 Table Small Collection, Stanford NER comparison of results for location between standard training set, and custom training set. topic 1 2 WORDS PROB WORDS PROB school 0.0175 days 0.0232 shooting 0.0085 ago 0.0159 elementary 0.0073 point2 0.0094 hook 0.0071 permalinksavecontextfull 0.0088 news 0.0068 2.0000 0.0076 sandy 0.0067 ago 0.0099 newtown 0.0067 you 0.0072 conneconnecticut 0.0057 I 0.0070 2012 0.0050 all 0.0056 from 0.0043 points1 0.0054 december 0.0041 reddit 0.0052 As visible from the results, this approach gave us a much better representation of the content of the collection when compared to using just word frequencies or POS tagging. The results obtained from our named entity recognition improved upon previous methods in providing unique indicative information for our collections. Table Small Collection, best topics Latent Dirichlet Allocation A topic model is a statistical model that aids in the discovering of abstract “topics” occurring in a collection . If a document is about a particular topic, we would expect that certain words would appear in the document more or less frequently. Latent Dirichlet Allocation (LDA) is a way of automatically
ShootingsReportDoc.docx