mouse genome informatics ( mgi ) is the primary international database resource for the laboratory mouse , providing integrated genetic , genomic and biological data to facilitate the study of human health and disease .
the mgi team curates the biomedical literature ( 11 000 publications per year ) and normalizes and integrates sequence and functional data about mouse genetics and genomics from almost 50 other external database and informatics resources .
mgi organizes curation teams around particular types of data including sequence data , phenotypes , embryonic expression data , comparative and functional information , mouse tumorigenesis and mouse models for human diseases .
mgi utilizes multiple bio - ontologies and is the authority for mouse gene and strain nomenclature .
the  mouse genome database ( 1 ) includes data on gene characterization , nomenclature , mapping , gene homologies among mammals , sequence links , phenotypes , disease models , allelic variants and mutants and strain data .
the  gene expression database ( 2 ) integrates different types of gene expression information from the mouse and provides a searchable index of published experiments on endogenous gene expression during development .
the  mouse tumor biology ( 3 ) database provides data on the frequency , incidence , genetics and pathology of neoplastic disorders , emphasizing data on tumors that develop characteristically in different genetically defined strains of mice .
the mgi group is a founding member of the  gene ontology consortium ( go , www.geneontology.org , ( 4 ) ) .
mgi fully incorporates the go in the database and provides a go browser for access to mouse functional annotation . finally , the  mousecyc database ( 5 ) focuses on  mus musculus metabolism and includes cell level processes such as biosynthesis , degradation , energy production and detoxification .
it is part of the biocyc collection of pathway databases created at sri international ( 6 ) .
here , we outline the workflow process for  one component of the mgi data acquisition and integration process  that associated with the  gene ontology project at mgi ( figure 1). mgi assigns functional annotations ( go terms ) to genes and protein products through semi - automated methods and manual curation .
semi - automated annotation strategies include mapping and translating data from the enzyme commission , swiss - prot , interpro ( see http://www.geneontology.org/go.indices.shtml ) , rat and human ortholog experimental data and others .
curation of these data sets includes review and resolution of quality control reports generated through the process of data loading and comparisons to existing data in mgi . for the purpose of this paper
, we will not discuss these semi - automated integration methods , but rather concentrate on the manual literature curation , which is a vital source of experimental mouse functional data .
while we focus on the go component of mgi in this description , the literature curation process is very similar for other mgi components that curate literature . 

figure 1.go curation workflow : papers of interest are identified and entered into the database system ( triage ) and associated with genes ( indexed ) .
go curation workflow : papers of interest are identified and entered into the database system ( triage ) and associated with genes ( indexed ) .
the subcomponents of the literature curation process include the following : 
 ( a)literature triage : identifying and obtaining relevant scientific literatureeach curator is assigned a specific subset of journals from a set of 160 relevant journals out of 650 subscriptions carried by the jackson laboratory .
journals are chosen for triage based on the numbers of articles that have been identified and curated for that journal over the previous years .
others are selected on the basis of particular annotation processes such as full curation of the wnt family of proteins , or in another example , full curation of genes implicated in human diseases . on a yearly basis
, the number of papers selected from all journals is tabulated , and the selection of which journals to regularly curate the next year is determined based on the relevancy of the journal publications during the previous year and the number of full time equivalents available for this task .
typically , a few journals are dropped and a few are added each year to the formal triage process .
the quosa application ( http://www.quosa.com ) is used as an aid to access and identify recent papers as represented in pubmed containing data about the mouse and determining which component of the database will be curated from the paper ( go , expression , mutant alleles , phenotypes , mapping , tumor ) .
quosa is used to retrieve full text pdfs for a journal issue or time frame of interest . because the experimental organism  mouse is often not mentioned in the abstract ( 334% depending on the journal )
, a curator selects an issue of a journal and then searches the full text of papers containing the keywords 
mice. the application highlights these terms showing the context of the search keywords , which enables curators to quickly determine whether the experiments described are of a suitable nature to be used for go annotation ( i.e. do the experiments aid in determining the normal function of the gene ? ) ( figure 2 ) . the number of papers examined and the number selected are somewhat journal dependent .
for example , out of 8090 papers per each weekly issue of journal of biological chemistry , roughly 60 contain one or more of the three keywords , and of those , the curator may select 1015 as being relevant for some area of the database . for an issue of nature , which may have 1520 research articles per issue , up to 5 may have the keywords and
papers selected are uploaded to an in - house server for the next steps.(b)adding the publications to the mgi system through the editorial interfacepapers selected as containing data appropriate for go annotation are entered into a  master bibliography module .
each record is tagged for the area of use of the database for which it has information .
information about journal , volume , pages and the abstract are automatically obtained nightly from pubmed using the pmid .
used box would then have an x added automatically.(c)indexing the papers to determine the genes being studieda paper is then associated or indexed to the genes discussed within by adding the paper to each gene s detail module ( figure 3b ) .
prominer , a natural language processing ( nlp ) application , is used to assist in the gene indexing process ( 7 ) .
utilizing official nomenclatures and synonym lists for mouse / rat / human gene names and gene symbols , prominer marks up papers for review by a curator , who then associates genes - to - papers in the mgi editorial interface ( ei ) ( figure 4 ) . 

figure 2.quosa information manager showing part of a paper from an issue of jbc with mouse , murine or mice highlighted .
curator quickly looks at context to select appropriate area of mgi that the paper best fits
figure 3.(a ) mgi master bib ei , showing a record for a paper that has been selected for alleles / phenotypes and go .
the paper has not yet been curated for go ( indicated by an x in the selected and x in not used ) .
references to be associated with this gene are entered into the lower left hand panel .
if a paper is associated with multiple genes , the same paper is entered for each gene .
literature triage : identifying and obtaining relevant scientific literature each curator is assigned a specific subset of journals from a set of 160 relevant journals out of 650 subscriptions carried by the jackson laboratory .
journals are chosen for triage based on the numbers of articles that have been identified and curated for that journal over the previous years .
others are selected on the basis of particular annotation processes such as full curation of the wnt family of proteins , or in another example , full curation of genes implicated in human diseases . on a yearly basis
, the number of papers selected from all journals is tabulated , and the selection of which journals to regularly curate the next year is determined based on the relevancy of the journal publications during the previous year and the number of full time equivalents available for this task .
typically , a few journals are dropped and a few are added each year to the formal triage process .
the quosa application ( http://www.quosa.com ) is used as an aid to access and identify recent papers as represented in pubmed containing data about the mouse and determining which component of the database will be curated from the paper ( go , expression , mutant alleles , phenotypes , mapping , tumor ) .
quosa is used to retrieve full text pdfs for a journal issue or time frame of interest .
because the experimental organism  mouse is often not mentioned in the abstract ( 334% depending on the journal ) , a curator selects an issue of a journal and then searches the full text of papers containing the keywords 
mice. the application highlights these terms showing the context of the search keywords , which enables curators to quickly determine whether the experiments described are of a suitable nature to be used for go annotation ( i.e. do the experiments aid in determining the normal function of the gene ? ) ( figure 2 ) .
for example , out of 8090 papers per each weekly issue of journal of biological chemistry , roughly 60 contain one or more of the three keywords , and of those , the curator may select 1015 as being relevant for some area of the database . for an issue of nature , which may have 1520 research articles per issue , up to 5 may have the keywords and
adding the publications to the mgi system through the editorial interface papers selected as containing data appropriate for go annotation are entered into a  master bibliography module .
each record is tagged for the area of use of the database for which it has information .
information about journal , volume , pages and the abstract are automatically obtained nightly from pubmed using the pmid .
for example , if this paper was used for a go annotation , the  used box would then have an x added automatically .
indexing the papers to determine the genes being studied a paper is then associated or indexed to the genes discussed within by adding the paper to each gene s detail module ( figure 3b ) .
prominer , a natural language processing ( nlp ) application , is used to assist in the gene indexing process ( 7 ) . utilizing official nomenclatures and synonym lists for mouse / rat / human gene names and gene symbols , prominer marks up papers for review by a curator , who then associates genes - to - papers in the mgi editorial interface ( ei ) ( figure 4 ) .
quosa information manager showing part of a paper from an issue of jbc with mouse , murine or mice highlighted .
curator quickly looks at context to select appropriate area of mgi that the paper best fits .
( a ) mgi master bib ei , showing a record for a paper that has been selected for alleles / phenotypes and go .
the paper has not yet been curated for go ( indicated by an x in the selected and x in not used ) .
references to be associated with this gene are entered into the lower left hand panel . if a paper is associated with multiple genes , the same paper is entered for each gene .
output from prominer : prominer is used to assist in the gene indexing process . utilizing official nomenclatures and synonym lists for mouse / rat / human gene names and gene symbols , prominer marks up papers for review by a curator , who then associates genes - to - papers in the mgi ei .
sometimes a paper discusses several genes , but not all of them may be objects for direct go annotation .
for example , a paper describing the effects of a knock out of a particular gene may use analysis of other gene products to analyze the particular processes being affected , but the annotation to involvement in the process would only be made to the gene being knocked out .
currently , the topical areas selected for each paper are not directly tied to the genes associated with the paper .
thus , a paper selected for go for one of the genes will appear in the go ei interface of unused papers for each of the genes indexed to the paper .
ideally , all papers would be immediately used for go annotation , but on average 300 new papers are added to the database each week , only less than half of these are curated each week due to resource limitations .
therefore , various priority selection criteria are used to choose which genes and papers warrant immediate attention
. reports of interest are generated , such as  genes with no go annotation but have new indexed literature selected for go or  genes with mutant alleles that have literature selected for go ( figure 5 ) .
additionally , participation in various collaborative projects , such as the reference genome project ( 8) , or the protein ontology ( 8) defines primary sets of genes to work based on community input . 

a curator at mgi uses the go ei module to enter annotations ( figure 6 ) .
curators use the annotation guidelines set forth by the gene ontology consortium ( http://www.geneontology.org/go.annotation.shtml ) .
it is divided into two main data entry sections : the annotation area and the annotation properties area .
a list of papers selected for go that are associated with this gene is shown in the lower right panel .
individual protein isoforms or modified forms can be indicated using annotation properties where an i d for a specific isoform can be indicated . after reading a paper
next , the reference number is added ( 2 ) , as well as the evidence code ( 3 ) .
if required by the type of evidence code , additional information is added to the inferred from column ( 4 ) .
once the annotation is saved , information about the cell type that the experiment was done in , or the specific isoform , or tissue , is entered into the annotation properties section ( 5 ) .
additional ontologies such as the cell ontology ( 9 ) , mouse adult ( 10 ) and embryonic anatomies ( 11 ) , protein ontology ( 12 ) , and psi - mod ( 13 ) are used in the properties fields .
several of these are used to supply an extension to the annotation which are used in the gene association file ( gaf ) ( 5 ) .
curators can use the obo - edit tool ( 14 ) to load multiple ontologies to aid in searching for appropriate terms , as well as viewing the chosen term in the context of the rest of the ontology ( figure 7 ) .
the go vocabulary is refreshed daily from the go site , and only current go terms can be used , otherwise data entry is prohibited . only reference identifiers previously entered into 
there are other mechanisms , such as data loads , that provide go annotation . in all cases , provenance is provided .
the go ei also has a built - in report generator that highlights words matching go terms found in the abstracts of papers selected for go as an aid to suggesting the type of information and evidence present in a paper ( figure 8) . 

figure 6.mgi go data entry module : the interface is divided into three main sections : go annotation , annotation properties and search and reference tracking .
drop - down menus display pick - lists of allowed entries in various fields ( evidence property , go qualifier and evidence codes ) . numbered areas : 1 ,
go i d entry ; 2 , reference entry ; 3 , evidence code entry ; 4 ,  inferred_from entry required for certain evidence codes and 5 , annotation properties entry . 

the far left panel shows the vocabularies that have been loaded for searching and viewing .
the right panel displays the terms in all of the vocabularies that contain the word 
kidney. the go term  kidney mesenchyme morphogenesis is selected and is visible as a tree view showing its children ( middle panel ) , and as a graphical view showing its parents ( lower left ) . 

figure 8.report generated using the abstracts of papers selected for go for the gene being annotated within the go ei .
mgi go data entry module : the interface is divided into three main sections : go annotation , annotation properties and search and reference tracking .
drop - down menus display pick - lists of allowed entries in various fields ( evidence property , go qualifier and evidence codes ) . numbered areas : 1 ,
go i d entry ; 2 , reference entry ; 3 , evidence code entry ; 4 ,  inferred_from entry required for certain evidence codes and 5 , annotation properties entry .
the far left panel shows the vocabularies that have been loaded for searching and viewing .
the right panel displays the terms in all of the vocabularies that contain the word 
kidney. the go term  kidney mesenchyme morphogenesis is selected and is visible as a tree view showing its children ( middle panel ) , and as a graphical view showing its parents ( lower left ) .
report generated using the abstracts of papers selected for go for the gene being annotated within the go ei .
annotations are tracked based on a variety of criteria such as annotation source ( mgi curation or data load ) and evidence ( experimental or predictive , such as through orthology or functional domain ) .
scripts review changes to the go structure and provide qc reports for curators noting genes whose annotations may be affected by these changes ( figure 9 ) .
additionally , we use the master bibliography tables and the go annotations to keep track of various areas that need focus , such as  genes with no go annotation but have papers that are selected for go but not used. 
 figure 9.go change log report showing changes to the go and genes with annotations using the term that may need to be looked at .
go change log report showing changes to the go and genes with annotations using the term that may need to be looked at .
all data assertions in mgi are supported by evidence and citation to the source of the information . for assertions that are associated with controlled vocabularies such as the go ,
links are provided to vocabulary browsers that provide the relationships between the assertion and other knowledge in that area of the ontology . using the table and associated information
mgi also provides a graphical display of go annotations from the go detail page for each gene . 

figure 10.the go annotation details for drd2 displayed as summary text ( a ) , table ( b ) or graph ( c ) .
the go annotation details for drd2 displayed as summary text ( a ) , table ( b ) or graph ( c ) .
go annotations are also shared with the go consortium ( goc ) through a gaf .
this is a tab - delimited file that contains most of the elements of a go annotation as outlined in the go ei section above .
presently , only the  cell type and  gene product annotation properties are included in the gaf .
this file is available on either the goc web site or along with other data sets , from the mgi ftp site ( ftp://ftp.informatics.jax.org/pub/reports/index.html ) .
the gaf and the go vocabulary file are used as input for many analytical tools such as go termfinder ( 15 ) .
instructions for construction of a go gaf file are found in go documentation at http://www.geneontology.org/go.format.annotation.shtml .
although some groups have used nlp to expedite the curation of literature ( 16 ) , this can be especially difficult to do when applied to mouse biology because of the integration of human and mouse studies within the same description of results .
the concepts captured by the go can not be gleaned just from simple text matching of terms , but must also take into account inferences that reflect a given context .
additionally , an understanding of the nature of an experimental assay is important to correctly use the information as evidence of a particular result . while we continue to work with nlp developers to design a system to automate identification and tagging of papers ( 17 ) ,
it is clear that the complexity of understanding the information in a biomedical publication requires the intervention of an experienced biologist - curator in the process .
funding for open access charge : mgi database resources are funded by grants from the national human genome research institute ( hg00330 , hg02273 ) , national institutes of health / national institute of child health and human development ( hd062499 ) and the national cancer institute ( ca89713 ) .
 the mouse genome database , the gene expression database and the mouse tumor biology database are integrated components of the mouse genome informatics ( mgi ) resource ( http://www.informatics.jax.org ) . 
 the mgi system presents both a consensus view and an experimental view of the knowledge concerning the genetics and genomics of the laboratory mouse . from genotype to phenotype , this information resource integrates information about genes , sequences , maps , expression analyses , alleles , strains and mutant phenotypes . 
 comparative mammalian data are also presented particularly in regards to the use of the mouse as a model for the investigation of molecular and genetic components of human diseases . 
 these data are collected from literature curation as well as downloads of large datasets ( swissprot , locuslink , etc . ) . 
 mgi is one of the founding members of the gene ontology ( go ) and uses the go for functional annotation of genes . here 
 , we discuss the workflow associated with manual go annotation at mgi , from literature collection to display of the annotations . 
 peer - reviewed literature is collected mostly from a set of journals available electronically . 
 selected articles are entered into a master bibliography and indexed to one of eight areas of interest such as  go or  homology or  
 phenotype. each article is then either indexed to a gene already contained in the database or funneled through a separate nomenclature database to add genes . 
 the master bibliography and associated indexing provide information for various curator - reports such as  papers selected for go that refer to genes with no go annotation. once indexed , curators who have expertise in appropriate disciplines enter pertinent information . 
 mgi makes use of several controlled vocabularies that ensure uniform data encoding , enable robust analysis and support the construction of complex queries . 
 these vocabularies range from pick - lists to structured vocabularies such as the go . 
 all data associations are supported with statements of evidence as well as access to source publications . 
Introduction
Curation triage: selecting what genes to annotate
Data entry: creating a GO annotation using the EI module for GO
Tracking metrics and quality control measures to set priorities for upcoming work
Annotation presentation and usage
Information access: NLP and beyond
Funding