A method and computer program product for implementing indexed natural language processing are disclosed. Source document features including but not limited to terms, punctuation, parts-of-speech, phrases (including the syntactic types of the phrases), dependent clauses (including the syntactic types of the dependent clauses), independent clauses (including the syntactic types of the independent clauses), sentences, paragraphs, labeled document sections and document type and cognitive grammar constraints on the scope of influence and binding for the same are entered into an index by their begin and end byte offsets (or some alternative indexing method). Queries against the source documents are implemented as nested constructs that specify queries as sets that have terms or other sets as set elements and where sets may be constructed according to: 1) ordering (or the lack thereof); 2) boolean relations; 3) fuzzy relations; and 4) scoping according to: a) proximity; b) phrase inclusion; c) clause inclusion; d) sentence inclusion; e) paragraph inclusion; f) section inclusion; g) document type; and cognitive grammar constraints. Further, terms that are the components of a query are divided into sets according to the expected cognitive grammar relations between those terms as they would appear as surface forms in the source documents. As an aid to constructing queries in this manner, in some implementations, a surface form ontology is implemented in which the surface forms from which desired concepts can be expressed are represented according to their cognitive grammar compositions. Using these methods, queries can be composed that analyze the source documents via the intermediary of an index at a level of detail that has heretofore been possible only by application of standard Natural Language Processing (NLP) techniques directly to the source document. This novel application combining the strengths of cognitive grammar, surface form ontology and indexing results in information retrieval (IR) with significantly improved levels of recall and precision and information extraction (IE) with significantly improved flexibility and processing speeds over very large sets of data.
Indexed Natural Language Processing