<title>Introduction</title> <p>Large-scale or systems-wide analysis of relationship networks (e.g. protein-protein) is built from individual links. Since labor is expensive, the domain of knowledge vast, and time short, automated methods of constructing these networks is of paramount importance in bioinformatics research [##REF##17977867##1##]. The simplest method of constructing such a network is by co-occurrence of terms, either in sentences or abstracts [##REF##11326270##2##, ####UREF##0##3##, ##REF##18321884##4####18321884##4##]. Previous research found that entities co-occurring within a sentence have approximately an 80% chance of being related in a non-trivial manner, while entities co-occurring within an abstract have approximately a 50% chance [##UREF##1##5##,##REF##14960466##6##] (exact numbers vary). However, these co-occurrence based approaches, despite their computational efficiency, necessarily remain agnostic about the nature of the relationship between entities. And if we are to have any hope at understanding how control and cause/effect are propagated in these networks, we must establish directionality.</p> <title>NLP systems and biological networks</title> <p>Establishing relationship directionality usually requires some means of natural language processing (NLP) to extract relationships from biomedical text, although it is not necessarily the only way [##REF##16438716##7##,##REF##18047707##8##]. Most systems that attempt to characterize the nature of the relationship between entities (e.g., directionality, stimulation, inhibition) use grammatical information from sentences, which is provided by NLP software. These may use either shallow parsing [##REF##16095962##9##], which divides the sentence into chunks, or deep parsing [##REF##17052900##10##,##REF##17134476##11##], which provides a complete representation of a sentence's constituent grammatical relations. For example, Chilibot [##REF##15473905##12##] and MedScan [##REF##15033866##13##, ####REF##12967967##14##, ##REF##16563163##15####16563163##15##] attempt to parse sentences to detect the nature of relationships and develop association networks. Chilibot has only three relationship types – positive, negative, and neutral – while MedScan has many more. Both Chilibot and MedScan attempt to determine which terms are subject and object. Systems can also be distinguished by whether they use grammatical information in a rule-based manner, like Chilibot, or whether they use machine learning techniques, such as SVMs [##UREF##2##16##].</p> <p>NLP-based approaches have seen heavy use in identifying protein-protein interactions within text [##REF##16873475##17##, ####REF##18315856##18##, ##REF##18207462##19##, ##REF##17238742##20##, ##REF##12799355##21##, ##REF##11262958##22##, ##REF##12689350##23####12689350##23##], gene-disease relationships [##REF##18433469##24##], and have been successful in bolstering and partially reconstructing regulatory networks using text [##REF##17683642##25##]. The goals for these programs range from a simplified two-class method of labeling extracted relationships (e.g., 'stimulates' (positive) or 'inhibits' (negative)), to detection of several dozen semantic types of relationship, such as 'binds', 'cleaves', 'phosphorylates', etc. Our interests in this line of research are to detect paths of directional inference (e.g. A increases B and B decreases C, therefore A decreases C) and their reliability. To accomplish this we must first develop a method that is scalable. Accuracy is desirable, of course, but if an algorithm has a modest accuracy yet many chances to detect directional relations, then it is possible the large sample size may offset a less-than-optimal accuracy. To do this, however, we need to better understand how these methods perform on large unstandardized corpora and the amount of apparently contradictory relationships that are detected as the amount of text is increased and false-positives accrue. Most approaches to date have unfortunately focused on relatively small corpora and it is not clear how they would perform on large datasets (e.g. millions of records).</p> <p>We anticipate some contradiction will be factual in nature (e.g. one author reporting A increases B and another reporting A decreases B), but most likely the primary form of contradiction will be context-dependent. That is, the context of the relationship may enable both scenarios to be true and therefore the different facts extracted from text only appear to be contradictory when examined in isolation. We also hypothesize that some objects will better serve as subjects of study, which will bias observations towards one type of directional relationship when, in theory, both might be considered equally logical. For example, the relationship between insulin and glucose will tend more towards reports on the extracellular effects of insulin on glucose (insulin decreases glucose) rather than the intracellular effects (insulin increases glucose) because the extracellular effects are much easier to assay for (e.g. by drawing blood). This is going to introduce an artifact into any NLP analysis that will eventually need to be resolved in the thrust towards a more "systems-level" view of biology. For now, however, we hope that such contextual biases will be relatively uniform (e.g. if insulin increases or decreases other entities, the same perspective biases will apply for them) so that making inferences might be possible.</p> <p>Here we present a directional relation extraction (DRE) system that uses a support vector machine (SVM) to classify dependency paths, as SVMs have been shown recently to be suited to this type of task [##REF##18003645##26##]. We term it the directional relation extraction and determination with SVM (DREAD SVM). It has the following notable features: First, it has a robust, extensible dependency path convolution kernel capable of extracting and determining directionality for relations when trained on the GENIA event corpus. Second, it has a thesaurus-based named entity recognition algorithm that is not limited to a particular type of biomedical entity (such as gene or protein). We then present the results of cross-validation on the GENIA event corpus and explore the results of large-scale extraction from MEDLINE abstracts. We observe that relationships are often extracted with a degree of ambiguity in their direction and nature – a phenomenon that is often viewed as inconvenient noise that is to be screened out during construction of biological networks. However, it is these instances we are most interested in for this report. The construction of biological networks often is centered upon the arrow and node (or vertices and edges) concept. Yet, how many of these relationships are of such a simple nature that they can be represented in such a manner and how many might be essentially irresolvable without a 3<sup>rd </sup>variable, such as perspective or time? We do not believe we can answer this conclusively here, but through the analysis of several terms expected to have relatively straightforward relationships, we hope to gain a better understanding of how often these simplified relationship constructs are valid.</p>
<title>Materials and methods</title> <p>The DREAD algorithm begins with the input of a sentence containing a term of interest, which must appear in a thesaurus of biomedical objects, including genes, chemicals, and clinical phenotypes (described below). The final output of the algorithm is a set of directed edges between the query term and all other terms that co-occurred with the query term within a sentence. These directed edges are also characterized for the presence of stimulatory or inhibitory interactions. In order to achieve this output, the DREAD algorithm goes through several stages of activity: namely Preprocessing, Parsing, and Relationship Resolution (Figure ##FIG##0##1##). Each stage has a corresponding broad function:</p> <p>• The <italic>Preprocessing </italic>phase formats the sentence for parsing by tokenizing and Named Entity Recognition (NER).</p> <p>• The <italic>Parsing </italic>phase uses Natural Language Processing tools to construct a grammatical representation of the sentence in Stanford Dependency (SD) format. The SD format casts words (tokens) as a network of nodes connected by grammatical <italic>dependencies </italic>(edges).</p> <p>• The <italic>Relationship Classification </italic>phase uses a combination of Support Vector Machines and heuristics to deduce the directionality and possible stimulatory or inhibitory character of a relationship based on the grammatical information provided by the parsing phase.</p> <title>Preprocessing</title> <p>The LingPipe software first breaks each sentence into a sequence of tokens <ext-link ext-link-type="uri" xlink:href="http://alias-i.com/lingpipe/"/>. Next, NER identifies biologically relevant terms within the sentence using a synonym dictionary of genes, chemicals, and clinical phenotypes developed as part of the IRIDESCENT system [##REF##14960466##6##,##REF##15471547##27##,##REF##14734310##28##]. Multiple-token terms are condensed to a single token in order to simplify the subsequent parsing steps. At this point, if the sentence does not contain at least the query term and one other named entity within the synonym dictionary, it is thrown out.</p> <title>Parsing</title> <title>Deep parsing</title> <p>The preprocessed sentences containing entities of interest are sent to the Charniak-Lease parser [##UREF##3##29##], which performs both part-of-speech tagging and deep parsing, returning a representation of the sentence in Penn Treebank format. The Charniak-Lease parser was previously found to be computationally efficient and accurate on biomedical text compared with other deep parsers [##REF##17254351##30##]. The Penn Treebank (PTB) representation of each sentence parsed is then stored in the local relational database awaiting evaluation by the next steps in the system (Figure ##FIG##1##2##).</p> <title>Conversion to Stanford Dependency format</title> <p>In order to distill the amount of grammatical information used as feature space for machine-learning-based relation extraction to a small, rich subset, we convert the Penn Treebank-format (PTB) output from the Charniak-Lease parser into the Stanford Dependency (SD) format. This SD format represents sentences as a network of nodes (words) and edges (grammatical relationships between words), as described in de Marneffe <italic>et al </italic>[##UREF##4##31##] (See Figure ##FIG##2##3##). In the DREAD system, the PTB-&gt;SD conversion is accomplished using a suite of tools available from Stanford; however, recent work has shown that conversion to typed dependencies from other formats, such as the HPSG output of the GENIA team's Enju parser, is also possible [##UREF##5##32##].</p> <title>Extraction of dependency path</title> <p>After converting the sentence to SD format, we further reduce the amount of grammatical information used in relation extraction by using only the "dependency path" – that is, the shortest sequence of nodes and edges between two entities in the grammatical network of typed dependencies (Figure ##FIG##3##4##). The DREAD system extracts a dependency path between the query term and each other term in the sentence that was identified by the dictionary-based NER step. If there are multiple instances of the query term within the sentence, this process is repeated, omitting dependency paths connecting the query term to another instance of itself.</p> <p>The dependency path has previously been shown to provide a good feature space for machine-learning-based relation extraction [##UREF##6##33##,##REF##17142812##34##]. Others, however, have used sub-trees of the syntactic parse tree (i.e., Penn Treebank format) or of the typed dependency format as feature space for relation extraction, to good effect [##UREF##7##35##]. A study of the optimal feature space for relation extraction is provided in Jiang and Zhai [##UREF##8##36##].</p> <title>Relationship classification</title> <p>In order for the DREAD system to draw a conclusion about the nature of a relationship based on the grammatical information provided by a dependency path, we use two sub-systems. The first, a pipeline of Support Vector Machines (SVMs), is used to establish the presence and directionality of a relationship for each dependency path. If the SVM classification procedure determines that the dependency path does indeed represent a directional relationship, the second system, a simple set of heuristic rules, classifies the relationship as "stimulatory" (+), "inhibitory" (-), or "neutral" (n). Together, these two pieces of information (directionality and stimulatory/inhibitory classification) are a summarized form of the interaction between two entities in the sentence (Figure ##FIG##4##5##).</p> <title>SVM-based directionality determination</title> <p>The SVM classification pipeline is composed of three one-against-one "convolution dependency path kernel" SVMs (described below). Each SVM has a training set derived by parsing and transforming the GENIA event corpus [##REF##12855455##37##,##REF##18182099##38##] into dependency paths. The three SVMs classify the dependency path in the following ways:</p> <p>1. Is there a functional relationship between the two entities?</p> <p>2. If there is a relationship, is that relationship directional?</p> <p>3. If directional, is the direction forward or reverse?</p> <p>If SVM #1 or #2 returns a negative answer (i.e., there is no relationship or the relationship is not directed), then the dependency path is not sent to any further steps.</p> <title>Convolution dependency path kernel</title> <p>Bunescu described the development of a SVM kernel that computes the similarity of two sequences of features by the number and length of common subsequences between them [##UREF##6##33##]. In that work, however, Bunescu used tokens, part-of-speech tags, and entity classes in the order that they appear within the sentence. As a concrete example, in comparing the sentences "I went to the store" and "You went to the game", the longest common subsequence of tokens would be "went...the", and other subsequences would be "went...to" and "to...the". In another work, Bunescu <italic>et al </italic>restricted feature classes to those within dependency paths, but simply calculated the number of co-occurring features as the kernel score [##UREF##9##39##]. This method had the limitation that paths with different lengths were computed as having zero similarity, which presumably lowered recall.</p> <p>We reasoned that the accuracy of the overall SVM procedure could be improved by combining the two methods; in other words, by restricting the set of features to those within the dependency path but by using Bunescu's subsequence kernel to compare two potential relationships with greater flexibility. Indeed, a recent study by Wang, who coined the name "convolution dependency path kernel", showed that combining the two methods leads to greater precision and recall [##UREF##10##40##].</p> <p>Formally, the generalized subsequence kernel is defined [##UREF##9##39##]:</p> <p></p> <p>Where <italic>u </italic>is a subsequence from the set of all possible sequences  subject to the constraint that <italic>u </italic>must appear as a subsequence in both of the feature sequences <italic>s </italic>and <italic>t </italic>(that is, <italic>u </italic>≺ s[i], u ≺ t[j]); <bold>i </bold>and <bold>j </bold>are the sequences of indices corresponding to <italic>s </italic>and <italic>t</italic>. Each subsequence <italic>u </italic>is penalized according to the decaying factor, λ, with the result that when the feature sequences <italic>s </italic>and <italic>t </italic>are longer, each subsequence is given less weight. This serves as a way of normalizing the kernel scores for feature sequences of different lengths.</p> <title>Feature selection</title> <p>There are three primary types of feature available in each dependency path for use by machine-learning classification: the token (word), the part-of-speech tag for each token, and the grammatical dependency (SD edge) connecting two tokens. We elect not to use the token as a feature; because MEDLINE abstracts consist of such a large vocabulary, choosing the token would likely lead to overfitting of the training data. On the other hand, both POS tags and grammatical dependencies provide a relatively sparse set of feature classes, and are more relevant to the grammatical structure of the sentence than the token itself.</p> <title>GENIA training corpus</title> <p>The GENIA event corpus consists of 1,000 abstracts from MEDLINE that have been manually annotated for various biomedical "events", largely molecular interactions. Of the events in this corpus involving two entities, some are directed, while others are not. These events have various annotations; we considered all relations of the <italic>Positive_regulation </italic>type as stimulatory, all relations of the <italic>Negative_regulation </italic>type as inhibitory, and all others as neutral. To derive gold standard relationships from the corpus, we first assembled all possible pair-wise combinations of entities within the sentence (because the GENIA event corpus is in a quasi-XML format, there are sometimes entities within entities; the broadest possible entity names were used). These pairs were then classified as follows:</p> <p>1. Exists vs. nonexistent – If an event was annotated between a pair of entities, a functional relationship between the two entities was said to exist; otherwise, the pair was classified as non-interacting.</p> <p>2. Directional vs. non-directional – Of the pairs of entities that were annotated as interacting, some relations showed one entity acting on the other (in GENIA parlance, this relationship had one "THEME" and one "CAUSE"), while other relationships did not show directionality (two "THEME" annotations). We divided interactions on this basis for the second training set.</p> <p>3. Forward vs. reverse – For directional relations, the "CAUSE" was considered to be the agent of the interaction, while the "THEME" was the target.</p> <p>To complete construction of the training corpus, we parsed all sentences and extracted dependency paths between each pair of entities as previously described. Dependency paths from MEDLINE at large could then be compared by the SVM kernel to dependency paths within the training corpus to classify the existence and directionality of relationships through a series of one-against-one classifiers.</p> <title>Heuristic determination of stimulatory or inhibitory relationships</title> <p>After using the pipeline of SVMs to establish the presence or absence of a relationship and, if applicable, its directionality, we apply a simple set of heuristic rules to determine whether the nature of the relationship described by the dependency path is "positive" (stimulation, induction, activation), "negative" (repression, down-regulation, deactivation), or "neutral" (none-of-the-above). This is done by matching stemmed tokens in the dependency path to a list of words in each category. We hope to soon characterize the stimulatory or inhibitory nature of relationships using a SVM [##UREF##2##16##]; however, initial efforts using tokens as features have resulted in good cross-validation accuracy albeit poor recall in large-scale MEDLINE extraction.</p> <title>Relationship resolution on large-scale datasets</title> <p>In order to test the performance of the DREAD algorithm on a large scale, and to test the hypothesis that aggregating relationship data from a large corpus will overcome deficiencies in precision on individual sentences, we sought out summary information for different objects in the database. We chose two common terms in the biomedical literature to focus on for analysis of unstructured text: A chemical, caffeine (17,145 papers with this term in the title and/or abstract) and gene, <italic>c-myc </italic>(11,971 papers). Summary information regarding the direction and nature of relationships between caffeine and other objects in the database was compiled from its Wikipedia entry <ext-link ext-link-type="uri" xlink:href="http://en.wikipedia.org"/>. For c-myc, we used NCBI's Gene Reference into Function (GeneRIF)[##REF##14728215##41##] instead because there was more information. Although Wikipedia may not always be appropriate as a source of summary information, we did feel that analysis of system performance should optimally be constrained to one or two standard sources of summary information so as not to bias the analysis (e.g., noticing that the system worked really well for a certain term and then hunting down supporting information related to that term while ignoring other terms that the system performed poorly on).</p> <p>After thus compiling a list of expected relationships between the two terms and other biomedical objects, we pulled all abstracts containing caffeine and <italic>c-myc </italic>from MEDLINE, split each abstract into sentences using the LingPipe utility, and preprocessed each sentence as described above. If the sentence was found to contain the target term, the DREAD system was used to predict the relationships between the target term and each other term in the sentence. If a directional relationship was found, it was categorized as either "forward" (F), indicating that caffeine or <italic>c-myc </italic>affects the other object, or as "reverse" (R), indicating that the other object acts upon caffeine or <italic>c-myc</italic>. "+" is defined as a stimulatory (e.g. increases, raises, upregulates, etc) relationship between two terms and "-" is an inhibitory relationship (e.g. decreases, lowers, inhibits, downregulates, etc.). For example, the relationship "caffeine increases cyclic AMP" would be notated "F+" from the perspective of caffeine.</p> <p>These relationships were then aggregated and each binary relationship was assigned a strength score based on the amount of contradiction. Extracted relations were considered contradictory when both stimulatory and inhibitory instances were detected for the same directionality. For example, finding A increases B and A decreases B results in a contradiction, but finding A increases B and B decreases A is not necessarily contradictory. Feedback and feedforward loops, for example, can accommodate such trends. Forward scores are calculated as max(F+, F-)/((F+)(F-)) and reverse scores as max(R+, R-)/((R+)(R-)).</p>
<title>Results</title> <title>SVM cross validation on GENIA corpus</title> <p>To benchmark the performance of the convolution dependency path kernel on standardized biomedical corpora, we subjected the GENIA training data for each of the three SVMs (see <italic>GENIA training corpus </italic>section) to 10-fold cross-validation. For detecting the existence of a relationship and directionality to the relationship, the precision/recall curves are shown in Figure ##FIG##5##6##. Contrasting with Küffner <italic>et al </italic>[##UREF##2##16##], the DREAD SVM's ability to detect the existence of a relationship achieved a maximum F-measure of <bold>0.879 </bold>(Küffner 0.849) at a precision level of <bold>0.820 </bold>(Küffner 0.827) and recall of <bold>0.948 </bold>(Küffner 0.872). The ability of our algorithm in directional relationship extraction (DRE) achieved a maximum F-measure of <bold>0.794 </bold>(Küffner 0.749) at a precision level of <bold>0.704 </bold>(Küffner 0.846) and recall of <bold>0.912 </bold>(Küffner 0.672). For directional relationships, the precision/recall curves are shown in Figure ##FIG##6##7##. F-measure scores were similar between detection of forward (0.861) and reverse (0.791) relationships.</p> <p>Note that these scores are based upon the evaluation of dependency paths in which the recognition of named entities is part of the corpus; therefore, they do not take into account the considerable loss in recall and slight loss in precision that results from a dictionary-based named entity approach. Also, in the DREAD system, these SVMs are arranged in a pipeline fashion, so that the error rate propagates in a multiplicative manner. Thus, these scores are a meaningful marker only of the performance of the SVM classifiers and not of the DREAD system as a whole.</p> <title>Method scalability</title> <p>For comparable tasks, DREAD SVM performance on the GENIA corpus was comparable to levels of precision and recall reported by Küffner <italic>et al </italic>[##UREF##2##16##]. This suggested that it was reasonable to proceed forward and examine how well performance would scale on much larger corpora.</p> <title>Caffeine analysis</title> <p>Using summary information for each of our target terms to know what relationships should be found, and the nature of those relationships, we ran the DREAD SVM on all MEDLINE abstracts with the target term. The same directional relationship had to be observed at least twice to be counted as a directional relationship prediction. Table ##TAB##0##1## summarizes the DREAD SVM performance on extracting directional relationships between caffeine and other objects in MEDLINE. Caffeine was chosen for three reasons: First, it was expected to have a predominance of stimulatory relationships. Second, it was expected to affect other objects in the body, but not be affected by many other objects (outside of metabolic pathways). And third, the authors have substantial first-hand experience with the physiological effects of this chemical compound. Table ##TAB##0##1## shows that most relationships are indeed forward (253 versus 116 reverse) and stimulatory (118 versus 44 inhibitory), as expected, but clearly not every extracted directional relationship was unambiguous. The system only identified 12/21 (57%) of the expected relationships. It did, however, correctly identify the directionality of 10/12 (83%) of the relationships.</p> <p>Drowsiness would naturally seem like a phenotype that should be lowered upon caffeine intake. However, caffeine withdrawal can cause drowsiness [##REF##15448977##42##] and since "caffeine withdrawal" was not present in the database, the shorter term was recognized as the subject instead.</p> <title>C-myc analysis</title> <p>The oncogene, c-myc is well studied and expected to both affect other objects and in turn be affected, as part of a genetic network. The mechanisms which sometimes govern genetic control, especially over a gene that is estimated to possibly regulate up to 15% of all genes[##REF##11443860##43##] are expected to complicate DRE. Table ##TAB##1##2## summarizes the results. The system identified 21/39 (53%) of the expected relationships, and of those, it correctly identified the directionality for 13/21 (62%). Examining some of the individual sentences to better understand the reasons for failure, it became evident that the natures of relationships for c-myc were less straightforward than for caffeine. For example, the role of c-myc as an oncogene is well known and it is upregulated in breast cancer, yet the system identified many more examples of breast cancer "stimulating" c-myc. This is because these examples are reported as correlative relationships (e.g., "in the breast cancer biopsy, c-myc was upregulated").</p> <p>Another example is JUN (aka c-jun). The actual functional relationship between c-myc and JUN is forward, negative: c-myc inhibits c-jun [##REF##18314492##44##]. However, because both c-myc and c-jun are oncogenes, they are usually mentioned in the context of being upregulated together. Yet another difficulty in the analysis of c-myc is that sentences often refer to an increase or decrease in the effects or properties (e.g. "oncogenic properties") of c-myc, rather than c-myc itself. Table ##TAB##2##3## shows examples of sentences whose DRE was incorrectly classified. There is an average of 0.11 stimulatory terms per c-myc:c-jun path, but only 0.03 inhibitory terms. So, in this case, at least, there seems to be a literature bias towards describing relationships as stimulatory rather than inhibitory. We examined the different interaction types to see if any were longer on average or contained more stimulatory or inhibitory keywords, finding that gene-gene candidate interaction paths were considerably shorter than gene-process paths, and contained fewer interaction keywords (see Table ##TAB##3##4##).</p>
<title>Discussion and conclusion</title> <p>Analysis of unstandardized text comes with many challenges for term recognition, synonym mapping, homonym resolution, etc. And directional relation detection is yet another challenge, not just in a technical sense of algorithmically detecting relations, but also in the sense of resolving apparently contradictory data that may arise. NLP approaches so far isolate facts from sentences and resolve relationships between two entities, which may not be sufficient to accurately reconstruct or model relationship networks. As we report here, extracted relationships range from those with strong support for one type and direction of relationship over another to apparently contradictory relationships, some of which are quickly resolved by a better understanding of context.</p> <p>We find that for these tasks involving directional relation extraction, it is important to have terms be as unambiguous as possible. Stress and metabolism, for example, have common meanings (i.e., psychological stress and the rate of energy consumption by an organism, respectively) and more specific physiological meanings (i.e., a state of increased responsiveness to stimulus and the process by which chemicals change form, respectively). One approach to increase DRE accuracy would entail employing methods to break down broad categories into more specific subcategories (e.g. oxidative stress, physiological stress, stress-related illness, etc.), as the recognition of term relationships was not the limiting factor in this analysis but rather the accurate recognition of full terms.</p> <p>We also find that noun modifiers frequently complicate thesaurus-based analysis of terms, as in the case of c-myc. To increase or decrease a gene in a biological sense would likely refer to its mRNA levels, protein expression levels or molecular activity (e.g. catalytic) levels. However, study of genetic effects often proceeds via an assessment of how known function is affected by manipulating the system and thus increasing and decreasing statements refer to the modifier rather than the gene (e.g. "A decreased gene-B-related apoptosis", "A increased geneB phosphorylation", etc). While NLP systems are capable, in theory, of resolving such sentences, this style of study and writing causes more false-positives as corpus size increases.</p> <p>Ultimately, the scientific community is going to want to move towards testing of extracted networks for concordance with observed behavior. But to do this, a greater incorporation of contextual or conditional information will become necessary, which may not be possible to represent in a single graph. Temporal effects, for example, would seem to require a graph for short-term and long-term effects. Additionally, some model organisms are manipulated to alter genetic behavior, and relationships that are true within a mutant strain may not be true outside it. Similarly, some interactions are conditional. Worse, some of these conditional interactions may show up as contradictory relationships merely because not enough is known to understand that "A increases B" and "A decreases B" are both true depending upon the context. All this unfortunately complicates analysis, but at the same time seems unavoidable. Networks may well need several additional parameters per connection to accommodate contextual information. In part, this type of NLP work may guide some of this along – when apparently contradictory information is found, it is possible to initiate a secondary search for contextual clues that could accurately predict when one relationship type would be true over another. In such cases, a computer need not necessarily "understand" the context, but simply be able to identify it algorithmically. In the future, we hope to develop more accurate metrics to identify when directional relationships require context resolution prior to their inclusion in a biological network.</p>
<title>Discussion and conclusion</title> <p>Analysis of unstandardized text comes with many challenges for term recognition, synonym mapping, homonym resolution, etc. And directional relation detection is yet another challenge, not just in a technical sense of algorithmically detecting relations, but also in the sense of resolving apparently contradictory data that may arise. NLP approaches so far isolate facts from sentences and resolve relationships between two entities, which may not be sufficient to accurately reconstruct or model relationship networks. As we report here, extracted relationships range from those with strong support for one type and direction of relationship over another to apparently contradictory relationships, some of which are quickly resolved by a better understanding of context.</p> <p>We find that for these tasks involving directional relation extraction, it is important to have terms be as unambiguous as possible. Stress and metabolism, for example, have common meanings (i.e., psychological stress and the rate of energy consumption by an organism, respectively) and more specific physiological meanings (i.e., a state of increased responsiveness to stimulus and the process by which chemicals change form, respectively). One approach to increase DRE accuracy would entail employing methods to break down broad categories into more specific subcategories (e.g. oxidative stress, physiological stress, stress-related illness, etc.), as the recognition of term relationships was not the limiting factor in this analysis but rather the accurate recognition of full terms.</p> <p>We also find that noun modifiers frequently complicate thesaurus-based analysis of terms, as in the case of c-myc. To increase or decrease a gene in a biological sense would likely refer to its mRNA levels, protein expression levels or molecular activity (e.g. catalytic) levels. However, study of genetic effects often proceeds via an assessment of how known function is affected by manipulating the system and thus increasing and decreasing statements refer to the modifier rather than the gene (e.g. "A decreased gene-B-related apoptosis", "A increased geneB phosphorylation", etc). While NLP systems are capable, in theory, of resolving such sentences, this style of study and writing causes more false-positives as corpus size increases.</p> <p>Ultimately, the scientific community is going to want to move towards testing of extracted networks for concordance with observed behavior. But to do this, a greater incorporation of contextual or conditional information will become necessary, which may not be possible to represent in a single graph. Temporal effects, for example, would seem to require a graph for short-term and long-term effects. Additionally, some model organisms are manipulated to alter genetic behavior, and relationships that are true within a mutant strain may not be true outside it. Similarly, some interactions are conditional. Worse, some of these conditional interactions may show up as contradictory relationships merely because not enough is known to understand that "A increases B" and "A decreases B" are both true depending upon the context. All this unfortunately complicates analysis, but at the same time seems unavoidable. Networks may well need several additional parameters per connection to accommodate contextual information. In part, this type of NLP work may guide some of this along – when apparently contradictory information is found, it is possible to initiate a secondary search for contextual clues that could accurately predict when one relationship type would be true over another. In such cases, a computer need not necessarily "understand" the context, but simply be able to identify it algorithmically. In the future, we hope to develop more accurate metrics to identify when directional relationships require context resolution prior to their inclusion in a biological network.</p>
<p>This is an open access article distributed under the terms of the Creative Commons Attribution License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/2.0"/>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</p> <title>Background</title> <p>Relationships between entities such as genes, chemicals, metabolites, phenotypes and diseases in MEDLINE are often directional. That is, one may affect the other in a positive or negative manner. Detection of causality and direction is key in piecing pathways together and in examining possible implications of experimental results. Because of the size and growth of biomedical literature, it is increasingly important to be able to automate this process as much as possible.</p> <title>Results</title> <p>Here we present a method of relation extraction using dependency graph parsing with SVM classification. We tested the SVM classifier first on gold standard corpora from GENIA and find it achieved 82% precision and 94.8% recall (F-measure of 87.9) on these standardized test sets. We then applied the entire system to all available MEDLINE abstracts for two target interactions with known effects. We find that while some directional relations are extracted with low ambiguity, others are apparently contradictory, at least when considered in an isolated context. When examined, it is apparent some are dependent upon the surrounding context (e.g. whether the relationship referred to short-term or long-term effects, or whether the focus was extracellular versus intracellular).</p> <title>Conclusion</title> <p>Thesaurus-based directional relation extraction can be done with reasonable accuracy, but is prone to false-positives on larger corpora due to noun modifiers. Furthermore, methods of resolving or disambiguating relationship context and contingencies are important for large-scale corpora.</p>
<title>Competing interests</title> <p>The authors declare that they have no competing interests.</p> <title>Authors' contributions</title> <p>JDW conceived of and supervised the project, CBG wrote the programs and implemented them. Both authors contributed to writing the manuscript.</p>
<title>Acknowledgements</title> <p>We would like to acknowledge funding support from NSF-EPSCoR grant EPS-0447262 and NIH/NLM grant 1 R01 LM009758-01.</p> <p>This article has been published as part of <italic>BMC Bioinformatics </italic>Volume 9 Supplement 9, 2008: Proceedings of the Fifth Annual MCBIOS Conference. Systems Biology: Bridging the Omics. The full contents of the supplement are available online at <ext-link ext-link-type="uri" xlink:href="http://www.biomedcentral.com/1471-2105/9?issue=S9"/></p>
<fig position="float" id="F1"><label>Figure 1</label><caption><p>Flowchart of the relation extraction process.</p></caption></fig> <fig position="float" id="F2"><label>Figure 2</label><caption><p>An example of sentence structure for the sentence "<italic>Both isobutylmethylxanthine and theophylline increased the level of cyclic AMP in rat mast cells</italic>.". Shown are the grammatical relationships diagrammed in the Penn Treebank format.</p></caption></fig> <fig position="float" id="F3"><label>Figure 3</label><caption><p>Grammatical sentence structure in dependency graph format.</p></caption></fig> <fig position="float" id="F4"><label>Figure 4</label><caption><p>Path of dependency between database terms.</p></caption></fig> <fig position="float" id="F5"><label>Figure 5</label><caption><p>SVM identification of directional relationships.</p></caption></fig> <fig position="float" id="F6"><label>Figure 6</label><caption><p>Precision/recall curve for A) Detecting relationships and B) Detecting directional relationships within the GENIA corpus.</p></caption></fig> <fig position="float" id="F7"><label>Figure 7</label><caption><p>Precision/recall curve for detecting A) Forward relationships and B) Reverse relationships in the GENIA corpus.</p></caption></fig>
<table-wrap position="float" id="T1"><label>Table 1</label><caption><p>Identifying directional relationships in MEDLINE for the chemical compound caffeine on the basis of summarized relationships after analysis of 17,145 abstracts.</p></caption><table frame="hsides" rules="groups"><thead><tr><td align="left"><bold>ObjectName</bold></td><td align="left"><bold>No rel.</bold></td><td align="left"><bold>ND Rel.</bold></td><td align="left"><bold>F+</bold></td><td align="left"><bold>F-</bold></td><td align="left"><bold>Fn</bold></td><td align="left"><bold>R+</bold></td><td align="left"><bold>R-</bold></td><td align="left"><bold>Rn</bold></td><td align="left"><bold>Expected</bold></td><td align="left"><bold>Extracted</bold></td></tr></thead><tbody><tr><td align="left">(-)-Adrenaline</td><td align="right">58</td><td align="right">22</td><td align="right">14</td><td align="right">4</td><td align="right">17</td><td align="right">0</td><td align="right">3</td><td align="right">17</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left">(-)-Dopa</td><td align="right">5</td><td align="right">2</td><td align="right">1</td><td align="right">0</td><td align="right">1</td><td align="right">1</td><td align="right">0</td><td align="right">2</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left">Cyclic AMP</td><td align="right">30</td><td align="right">9</td><td align="right">21</td><td align="right">11</td><td align="right">12</td><td align="right">4</td><td align="right">3</td><td align="right">13</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left">diuresis</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left"><bold>
                                 <italic>Drowsiness</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>3</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="left"><bold><italic>F</italic>-</bold></td><td align="left"><bold>
                                 <italic>F+</italic>
                              </bold></td></tr><tr><td align="left">Fatigue</td><td align="right">23</td><td align="right">8</td><td align="right">4</td><td align="right">6</td><td align="right">15</td><td align="right">1</td><td align="right">0</td><td align="right">11</td><td align="left">F-</td><td align="left">F-</td></tr><tr><td align="left">Fatty acid</td><td align="right">10</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">6</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left">gastric acid secretion</td><td align="right">3</td><td align="right">3</td><td align="right">1</td><td align="right">0</td><td align="right">2</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left"><bold>
                                 <italic>Glycerol</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>15</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>6</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>3</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>8</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>3</italic>
                              </bold></td><td align="left"><bold>
                                 <italic>F+</italic>
                              </bold></td><td align="left"><bold><italic>F</italic>-</bold></td></tr><tr><td align="left">insomnia</td><td align="right">16</td><td align="right">3</td><td align="right">3</td><td align="right">0</td><td align="right">5</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left">Irritability</td><td align="right">2</td><td align="right">2</td><td align="right">2</td><td align="right">1</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left">Lethargy</td><td align="right">2</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="left">F-</td><td align="left">None</td></tr><tr><td align="left">lipid metabolism</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left">Nervousness</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">1</td><td align="right">5</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left">Norepinephrine</td><td align="right">61</td><td align="right">19</td><td align="right">10</td><td align="right">2</td><td align="right">18</td><td align="right">4</td><td align="right">0</td><td align="right">16</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left">Palpitation</td><td align="right">5</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left">PKA</td><td align="right">3</td><td align="right">1</td><td align="right">1</td><td align="right">0</td><td align="right">3</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left">Respiratory alkalosis</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left">ryanodine-sensitive calcium-release channel activity</td><td align="right">48</td><td align="right">18</td><td align="right">33</td><td align="right">2</td><td align="right">17</td><td align="right">5</td><td align="right">4</td><td align="right">15</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left">vasodilation</td><td align="right">2</td><td align="right">0</td><td align="right">1</td><td align="right">2</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="left">F-</td><td align="left">F-</td></tr><tr><td align="left">vasoconstriction</td><td align="right">5</td><td align="right">0</td><td align="right">3</td><td align="right">0</td><td align="right">2</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="left">F+</td><td align="left">F+</td></tr></tbody></table></table-wrap> <table-wrap position="float" id="T2"><label>Table 2</label><caption><p>Identifying directional relationships in MEDLINE for the gene c-myc.</p></caption><table frame="hsides" rules="groups"><thead><tr><td align="left"><bold>ObjectName</bold></td><td align="left"><bold>No rel.</bold></td><td align="left"><bold>ND Rel.</bold></td><td align="left"><bold>F+</bold></td><td align="left"><bold>F-</bold></td><td align="left"><bold>Fn</bold></td><td align="left"><bold>R+</bold></td><td align="left"><bold>R-</bold></td><td align="left"><bold>Rn</bold></td><td align="left"><bold>Expected</bold></td><td align="left"><bold>Extracted</bold></td></tr></thead><tbody><tr><td align="left"><italic>
                                 <underline>Process</underline>
                              </italic></td><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td align="left">Anaplasia</td><td align="right">6</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">1</td><td align="right">0</td><td align="right">4</td><td align="left">F+/R+</td><td align="left">None</td></tr><tr><td align="left"><bold>
                                 <italic>Angiogenesis</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>11</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>10</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>7</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>3</italic>
                              </bold></td><td align="left"><bold>
                                 <italic>F+</italic>
                              </bold></td><td align="left"><bold><italic>F</italic>-</bold></td></tr><tr><td align="left"><bold>
                                 <italic>Breast cancer</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>92</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>12</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>44</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>9</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>40</italic>
                              </bold></td><td align="left"><bold>
                                 <italic>F+</italic>
                              </bold></td><td align="left"><bold>
                                 <italic>R+</italic>
                              </bold></td></tr><tr><td align="left">Cell growth</td><td align="right">131</td><td align="right">34</td><td align="right">19</td><td align="right">19</td><td align="right">76</td><td align="right">6</td><td align="right">8</td><td align="right">34</td><td align="left">F+</td><td align="left">Inc.</td></tr><tr><td align="left">Cervical carcinoma</td><td align="right">22</td><td align="right">4</td><td align="right">0</td><td align="right">0</td><td align="right">3</td><td align="right">0</td><td align="right">0</td><td align="right">8</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left"><bold>
                                 <italic>DNA Damage</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>33</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>7</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>14</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>3</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>11</italic>
                              </bold></td><td align="left"><bold>
                                 <italic>F+</italic>
                              </bold></td><td align="left">R+</td></tr><tr><td align="left">DNA repair</td><td align="right">13</td><td align="right">11</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">1</td><td align="right">0</td><td align="right">4</td><td align="left">F-</td><td align="left">None</td></tr><tr><td align="left">Medulloblastoma</td><td align="right">21</td><td align="right">6</td><td align="right">4</td><td align="right">0</td><td align="right">13</td><td align="right">1</td><td align="right">0</td><td align="right">20</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left">Tumorigenesis</td><td align="right">80</td><td align="right">40</td><td align="right">20</td><td align="right">6</td><td align="right">48</td><td align="right">6</td><td align="right">4</td><td align="right">32</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td align="left"><italic>
                                 <underline>Chemical</underline>
                              </italic></td><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td align="left">Rapamycin</td><td align="right">4</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">2</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="left">R+</td><td align="left">None</td></tr><tr><td align="left">Reactive oxygen species</td><td align="right">9</td><td align="right">5</td><td align="right">2</td><td align="right">0</td><td align="right">4</td><td align="right">1</td><td align="right">1</td><td align="right">10</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left">Valproic acid</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="right">0</td><td align="left">R-</td><td align="left">R-</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td align="left"><italic>
                                 <underline>Gene</underline>
                              </italic></td><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td align="left">AICDA</td><td align="right">4</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="left">R+</td><td align="left">None</td></tr><tr><td align="left">AURKA</td><td align="right">5</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="left">R+</td><td align="left">None</td></tr><tr><td align="left">Calcineurin</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="left">R+</td><td align="left">None</td></tr><tr><td align="left"><bold>
                                 <italic>CDKN2A</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>95</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>37</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>5</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>26</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>7</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>42</italic>
                              </bold></td><td align="left"><bold><italic>F</italic>-</bold></td><td align="left"><bold><italic>R</italic>-</bold></td></tr><tr><td align="left">CREBBP</td><td align="right">7</td><td align="right">3</td><td align="right">0</td><td align="right">0</td><td align="right">5</td><td align="right">1</td><td align="right">0</td><td align="right">6</td><td align="left">R-</td><td align="left">None</td></tr><tr><td align="left">EPHA2</td><td align="right">8</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="left">F-</td><td align="left">None</td></tr><tr><td align="left"><bold>
                                 <italic>FBXW7</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>18</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>5</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>4</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>4</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>17</italic>
                              </bold></td><td align="left"><bold><italic>R</italic>-</bold></td><td align="left">R+</td></tr><tr><td align="left">HDAC1</td><td align="right">1</td><td align="right">5</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">4</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left">HMGCS2</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="left">F-</td><td align="left">None</td></tr><tr><td align="left">IFN-gamma</td><td align="right">2</td><td align="right">1</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">1</td><td align="left">R-</td><td align="left">None</td></tr><tr><td align="left"><bold>
                                 <italic>JUN</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>558</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>28</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>12</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>71</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>12</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>6</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>75</italic>
                              </bold></td><td align="left"><bold><italic>F</italic>-</bold></td><td align="left">F+/R+</td></tr><tr><td align="left">NDRG2</td><td align="right">2</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">5</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="left">F-</td><td align="left">None</td></tr><tr><td align="left">NFATC1</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">2</td><td align="right">0</td><td align="right">1</td><td align="left">R+</td><td align="left">R+</td></tr><tr><td align="left">PCGF2</td><td align="right">4</td><td align="right">4</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="left">R-</td><td align="left">None</td></tr><tr><td align="left">PPARG</td><td align="right">1</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left">PRL</td><td align="right">15</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">5</td><td align="right">6</td><td align="right">1</td><td align="right">5</td><td align="left">R+</td><td align="left">R+</td></tr><tr><td align="left"><bold>
                                 <italic>RAC1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>5</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>3</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="left"><bold>
                                 <italic>R+</italic>
                              </bold></td><td align="left">F+</td></tr><tr><td align="left">SP1</td><td align="right">29</td><td align="right">5</td><td align="right">0</td><td align="right">3</td><td align="right">9</td><td align="right">4</td><td align="right">0</td><td align="right">9</td><td align="left">R+</td><td align="left">R+</td></tr><tr><td align="left"><bold>
                                 <italic>STAT5A</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>6</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>1</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>2</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>0</italic>
                              </bold></td><td align="right"><bold>
                                 <italic>4</italic>
                              </bold></td><td align="left"><bold><italic>R</italic>-</bold></td><td align="left">R+</td></tr><tr><td align="left">Telomerase</td><td align="right">27</td><td align="right">5</td><td align="right">13</td><td align="right">0</td><td align="right">9</td><td align="right">5</td><td align="right">1</td><td align="right">8</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left">VEGFA</td><td align="right">49</td><td align="right">2</td><td align="right">0</td><td align="right">1</td><td align="right">18</td><td align="right">0</td><td align="right">0</td><td align="right">17</td><td align="left">F+</td><td align="left">None</td></tr><tr><td align="left">WRN</td><td align="right">1</td><td align="right">1</td><td align="right">2</td><td align="right">0</td><td align="right">3</td><td align="right">0</td><td align="right">0</td><td align="right">2</td><td align="left">F+</td><td align="left">F+</td></tr><tr><td align="left">ZBTB16</td><td align="right">3</td><td align="right">2</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="left">R-</td><td align="left">None</td></tr><tr><td align="left">ZBTB17</td><td align="right">24</td><td align="right">17</td><td align="right">1</td><td align="right">8</td><td align="right">11</td><td align="right">1</td><td align="right">3</td><td align="right">9</td><td align="left">F-</td><td align="left">F-</td></tr><tr><td align="left">Zfp472</td><td align="right">1</td><td align="right">2</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">0</td><td align="right">1</td><td align="left">R-</td><td align="left">None</td></tr></tbody></table></table-wrap> <table-wrap position="float" id="T3"><label>Table 3</label><caption><p>Example sentences from the failed DRE between c-myc and the entities "Breast Cancer" and "c-jun". The expected relationships for breast cancer was F+ and for c-jun, F-.</p></caption><table frame="hsides" rules="groups"><thead><tr><td align="left">Object</td><td align="left">Extracted Rel.</td><td align="left">PubMed ID</td><td align="left">Sentence</td></tr></thead><tbody><tr><td align="left">Breast Cancer</td><td align="left">R+</td><td align="right">12150449</td><td align="left">The c-myc oncogene is frequently activated in invasive breast cancer and has been associated with high nuclear grade, lymph node metastasis and poorer disease outcome</td></tr><tr><td/><td align="left">R+</td><td align="right">7734397</td><td align="left">The proto-oncogene c-myc is involved in the stimulation of cell proliferation, and its expression is known to be stimulated by estradiol (E2) in human breast cancer cell lines and various non – cancerous E2 – dependent tissues</td></tr><tr><td/><td align="left">F+</td><td align="right">1855215</td><td align="left">In search of critical genes in the mechanism of estrogen action in human breast cancer, we previously showed that estrogen stimulates transcription of the c – myc gene in estrogen-dependent (MCF-7) cells</td></tr><tr><td/><td/><td/><td/></tr><tr><td align="left">c-Jun</td><td align="left">R+</td><td align="right">8417822</td><td align="left">17 beta – Estradiol had little effect on expression of c-jun, jun B, jun D, or c-fos mRNA by MCF-7 cells over 12 h, although it stimulated c-myc expression 4-fold within 30 min</td></tr><tr><td/><td align="left">F+</td><td align="right">14523011</td><td align="left">Furthermore, we identify a phylogenetically conserved AP-1-responsive element in the promoter of the c-myc proto-oncogene that recruits in vivo the c-Jun and JunD AP-1 family members and controls the PDGF-dependent transactivation of the c-myc promoter</td></tr><tr><td/><td align="left">R-</td><td align="right">8219202</td><td align="left">In addition, intracellularly, mitoxantrone-induced PCD was associated with a marked induction of c-jun and significant repression of c-myc and BCL-2 oncogenes</td></tr></tbody></table></table-wrap> <table-wrap position="float" id="T4"><label>Table 4</label><caption><p>Path lengths and quantity of stimulatory and inhibitory tokens seem to vary with the type of object.</p></caption><table frame="hsides" rules="groups"><thead><tr><td/><td align="center"><bold>Number of Paths</bold></td><td align="center"><bold>Average Path Length (tokens)</bold></td><td align="center"><bold>Average stimulatory tokens per path</bold></td><td align="center"><bold>Average inhibitory tokens per path</bold></td></tr></thead><tbody><tr><td align="center"><bold>Myc:chemical</bold></td><td align="center">42</td><td align="center">7.64</td><td align="center">0.36</td><td align="center">0.21</td></tr><tr><td align="center"><bold>Myc:process</bold></td><td align="center">850</td><td align="center">7.23</td><td align="center">0.16</td><td align="center">0.11</td></tr><tr><td align="center"><bold>Myc:gene</bold></td><td align="center">1378</td><td align="center">4.93</td><td align="center">0.15</td><td align="center">0.06</td></tr></tbody></table></table-wrap>
<disp-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M1" name="1471-2105-9-S9-S11-i1" overflow="scroll">
                        <mml:semantics>
                           <mml:mrow>
                              <mml:msub>
                                 <mml:mi>K</mml:mi>
                                 <mml:mi>n</mml:mi>
                              </mml:msub>
                              <mml:mo stretchy="false">(</mml:mo>
                              <mml:mi>s</mml:mi>
                              <mml:mo>,</mml:mo>
                              <mml:mi>t</mml:mi>
                              <mml:mo>,</mml:mo>
                              <mml:mi>λ</mml:mi>
                              <mml:mo stretchy="false">)</mml:mo>
                              <mml:mo>=</mml:mo>
                              <mml:mstyle displaystyle="true">
                                 <mml:munder>
                                    <mml:mo>∑</mml:mo>
                                    <mml:mrow>
                                       <mml:mi>u</mml:mi>
                                       <mml:mo>∈</mml:mo>
                                       <mml:msubsup>
                                          <mml:mi>Σ</mml:mi>
                                          <mml:mo>∪</mml:mo>
                                          <mml:mi>n</mml:mi>
                                       </mml:msubsup>
                                    </mml:mrow>
                                 </mml:munder>
                                 <mml:mrow>
                                    <mml:mstyle displaystyle="true">
                                       <mml:munder>
                                          <mml:mo>∑</mml:mo>
                                          <mml:mrow>
                                             <mml:mstyle mathvariant="bold" mathsize="normal">
                                                <mml:mi>i</mml:mi>
                                             </mml:mstyle>
                                             <mml:mo>:</mml:mo>
                                             <mml:mi>u</mml:mi>
                                             <mml:mo>≺</mml:mo>
                                             <mml:mi>s</mml:mi>
                                             <mml:mo stretchy="false">[</mml:mo>
                                             <mml:mstyle mathvariant="bold" mathsize="normal">
                                                <mml:mi>i</mml:mi>
                                             </mml:mstyle>
                                             <mml:mo stretchy="false">]</mml:mo>
                                          </mml:mrow>
                                       </mml:munder>
                                       <mml:mrow>
                                          <mml:mstyle displaystyle="true">
                                             <mml:munder>
                                                <mml:mo>∑</mml:mo>
                                                <mml:mrow>
                                                   <mml:mstyle mathvariant="bold" mathsize="normal">
                                                      <mml:mi>j</mml:mi>
                                                   </mml:mstyle>
                                                   <mml:mo>:</mml:mo>
                                                   <mml:mi>u</mml:mi>
                                                   <mml:mo>≺</mml:mo>
                                                   <mml:mi>t</mml:mi>
                                                   <mml:mo stretchy="false">[</mml:mo>
                                                   <mml:mstyle mathvariant="bold" mathsize="normal">
                                                      <mml:mi>j</mml:mi>
                                                   </mml:mstyle>
                                                   <mml:mo stretchy="false">]</mml:mo>
                                                </mml:mrow>
                                             </mml:munder>
                                             <mml:mrow>
                                                <mml:msup>
                                                   <mml:mi>λ</mml:mi>
                                                   <mml:mrow>
                                                      <mml:mi>l</mml:mi>
                                                      <mml:mo stretchy="false">(</mml:mo>
                                                      <mml:mstyle mathvariant="bold" mathsize="normal">
                                                         <mml:mi>i</mml:mi>
                                                      </mml:mstyle>
                                                      <mml:mo stretchy="false">)</mml:mo>
                                                      <mml:mo>+</mml:mo>
                                                      <mml:mi>l</mml:mi>
                                                      <mml:mo stretchy="false">(</mml:mo>
                                                      <mml:mstyle mathvariant="bold" mathsize="normal">
                                                         <mml:mi>j</mml:mi>
                                                      </mml:mstyle>
                                                      <mml:mo stretchy="false">)</mml:mo>
                                                   </mml:mrow>
                                                </mml:msup>
                                             </mml:mrow>
                                          </mml:mstyle>
                                       </mml:mrow>
                                    </mml:mstyle>
                                 </mml:mrow>
                              </mml:mstyle>
                           </mml:mrow>
                           
                        </mml:semantics>
                     </mml:math></disp-formula> <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M2" name="1471-2105-9-S9-S11-i2" overflow="scroll"><mml:semantics><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mi>u</mml:mi><mml:mo>∈</mml:mo><mml:msubsup><mml:mi>Σ</mml:mi><mml:mo>∪</mml:mo><mml:mi>n</mml:mi></mml:msubsup><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:semantics></mml:math></inline-formula>
<table-wrap-foot><p>Possible directional relationships are classified as <bold><underline>F</underline></bold>orward (caffeine affects the object) or <bold><underline>R</underline></bold>everse (the object affects caffeine) and as stimulatory (+), inhibitory (-) or neutral (n). No rel = no relationship found, N.D. rel = no directional relationship found, Expected = the directional relation suggested by the summary information, Extracted = the directional relation with the most support, based on the sentences processed. <bold><italic>Bold italic </italic></bold>font indicates errors.</p></table-wrap-foot> <table-wrap-foot><p>See Table 1 for header explanations. ("Inc." = inconclusive - tied for the highest relation score).</p></table-wrap-foot> <table-wrap-foot><p>See Table 2 for the particular objects tested.</p></table-wrap-foot>
<graphic xlink:href="1471-2105-9-S9-S11-1"/> <graphic xlink:href="1471-2105-9-S9-S11-2"/> <graphic xlink:href="1471-2105-9-S9-S11-3"/> <graphic xlink:href="1471-2105-9-S9-S11-4"/> <graphic xlink:href="1471-2105-9-S9-S11-5"/> <graphic xlink:href="1471-2105-9-S9-S11-6"/> <graphic xlink:href="1471-2105-9-S9-S11-7"/>
[{"surname": ["Pratt", "Yetisgen-Yildiz"], "given-names": ["W", "M"], "article-title": ["LitLinker: Capturing Connections across the Biomedical Literature"], "source": ["Proceedings of the International Conference on Knowledge Capture (K-Cap'03): 2003; Florida"], "year": ["2003"], "fpage": ["105"], "lpage": ["112"]}, {"surname": ["Ding", "Berleant", "Nettleton", "Wurtele"], "given-names": ["J", "D", "D", "E"], "article-title": ["Mining Medline: Abstracts, Sentences or Phrases?"], "source": ["Pacific Symposium in Biocomputing: 2002; Kauau, Hawaii"], "year": ["2002"], "fpage": ["326"], "lpage": ["337"]}, {"surname": ["Kuffner", "Duchrow", "Fundel", "Zimmer"], "given-names": ["R", "T", "K", "R"], "article-title": ["Characterization of Protein Interactions"], "source": ["German Conference on Bioinformatics: 2006"], "year": ["2006"]}, {"surname": ["Lease", "Charniak"], "given-names": ["M", "E"], "article-title": ["Parsing Biomedical Literature"], "source": ["Natural Language Processing-IJCNLP: Second International Joint Conference: October 11\u201313 2005; Jeju Island, Korea"], "year": ["2005"]}, {"surname": ["de Marneffe", "MacCartney", "Manning"], "given-names": ["M-C", "B", "C"], "article-title": ["Generating Typed Dependency Parses from Phrase Structure Parses"], "source": ["Language Resources and Evaluation Conference: 2006"], "year": ["2006"]}, {"surname": ["Sagae", "Miyao", "Tsujii"], "given-names": ["K", "Y", "J"], "article-title": ["Challenges in Mapping of Syntactic Representations for Framework-Independent Parser Evaluation"], "source": ["Proceedings of the Workshop on Automated Syntatic Annotations for Interoperable Language Resources at the First International Conference on Global Interoperability for Language Resources (ICGL'08): 2008; Hong Kong"], "year": ["2008"]}, {"surname": ["Bunescu", "Mooney"], "given-names": ["RC", "RJ"], "article-title": ["A shortest path dependency kernel for relation extraction"], "source": ["Proceedings of the Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing: 2005"], "year": ["2005"], "fpage": ["724"], "lpage": ["731"]}, {"surname": ["Culotta", "Sorensen"], "given-names": ["A", "J"], "article-title": ["Dependency tree kernels for relation extraction"], "source": ["Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04): 2004"], "year": ["2004"], "fpage": ["423"], "lpage": ["429"]}, {"surname": ["Jiang", "Zhai"], "given-names": ["J", "C"], "source": ["A Systematic Exploration of the literature Space for Relation Extraction"], "year": ["2007"], "publisher-name": ["Rochester, New York: Association for Computational Linguistics"], "fpage": ["113"], "lpage": ["120"]}, {"surname": ["Bunescu", "Mooney"], "given-names": ["R", "RJ"], "article-title": ["Subsequence Kernels for Relation Extraction"], "source": ["The 19th Conference on Neural Information Processing Systems (NIPS): 2005; Vancouver, BC"], "year": ["2005"]}, {"surname": ["Wang"], "given-names": ["M"], "article-title": ["A Re-examination of Dependency Path Kernels for Relation Extraction"], "source": ["IJCNLP: 2008"], "year": ["2008"]}]
BMC Bioinformatics. 2008 Aug 12; 9(Suppl 9):S11