in recent years , targeted 
 proteomics has emerged as a mainstream 
 technology to detect and quantify protein and peptide abundances in 
 complex biological samples .
this popularity has , in part , 
 been fueled by the introduction and broad adoption of the enabling 
 skyline software tool .
skyline is a freely available 
 windows client application that has become a valuable tool for building 
 and optimizing selected reaction monitoring ( srm ) assays .
skyline 
 works across instrument platforms , and its intuitive user interface 
 and rich set of graphs simplifies the development and refinement of 
 targeted methods and also makes it easy for users to visually assess 
 data quality .
in addition to being the most commonly used tool for 
 analyzing data acquired on triple quadrupole instruments , skyline 
 is now widely used for data analysis and method development in experiments 
 that involve tof and ion trap mass spectrometers , spanning the techniques 
 of data - dependent acquisition ( dda ) , targeted 
 ms / ms ( also referred to as parallel reaction 
 monitoring or prm ) , and data - independent 
 acquisition dia .
the growth in adoption of skyline 
 for targeted proteomics experiments has resulted in an increasing 
 accumulation of skyline documents across various laboratories and 
 organizations .
these documents represent a large body of work to develop 
 targeted assays for precise quantitative measurement of proteins and 
 peptides across a range of experimental protocols , which have the 
 potential to become a resource of experimental information .
organizing 
 these documents into a database is essential to make it possible to 
 review and explore larger sets of well - annotated experimental data 
 in aggregate and to enable use of the accumulated knowledge to inform 
 targeted measurement in future experiments .
panorama was developed 
 with the goals of data integrity , security , 
 sharing , scalability , and ease - of - use .
it allows laboratories and 
 organizations to aggregate curated results contained in skyline documents , 
 without loss of information .
the panorama schema provides a relational 
 database storage format for the complete skyline document data model . 

panorama is designed both to facilitate sharing results among lab 
 members , external collaborators , and with the general public and to 
 keep private data secure during multiple experiment studies and the 
 peer - review process .
the rich security model of labkey server combined 
 with its support for installation on private servers , as well as the 
 availability of professional paid support , has allowed panorama adoption 
 even among large pharmaceutical companies .
panorama has been tightly 
 integrated with skyline , allowing users to publish their documents 
 to panorama directly from skyline and also to query results exported 
 from panorama when designing new methods in skyline ( figure 1 ) .
users publish documents 
 containing 
 refined targeted methods from skyline to a panorama server , where 
 they are stored in a relational database .
data uploaded to panorama 
 can be viewed , searched and analyzed in a web - browser interface .
documents 
 published to panorama as well as chromatogram libraries exported from 
 library folders can be downloaded to view and use in skyline .
existing repositories , such as 
 the peptide atlas srm experiment library ( passel ) and mrmaid - db , provide a place 
 for collecting and exploring experimental srm data and transitions 
 that have already been published .
these repositories are public and 
 do not provide mechanisms for secure data sharing with internal and 
 external collaborators prior to publication .
they also lack a local 
 installation option for organizations and laboratories that may prefer 
 to manage their own repositories .
they require the submission of raw 
 data and transition lists and do not yet have support for importing 
 the rich information about the data curation and analysis process 
 or custom annotations available in skyline documents .
in contrast , 
 panorama supports a knowledge base environment for continued interaction 
 with and analysis of large sets of targeted proteomics data in public 
 or private circles .
knowledge accumulated over time can be used to 
 create chromatogram libraries tailored to specific biological and 
 experimental conditions .
these libraries can be used to select the 
 most suitable peptides and product ions to monitor in a targeted experiment , 
 and the stored chromatograms may also be used to increase confidence 
 in chromatogram peak picking .
panorama has been implemented as a module in labkey server , a bioinformatics data management platform with 
 rich support for proteomics data .
both labkey server and the panorama 
 module are available as open - source software under the apache 2.0 
 license .
labkey server is a web application that is implemented in java 
 and runs on the apache tomcat web server .
it comes with core customizable 
 features for file management , data organization and security , role - based 
 permissions , integration with analysis tools like r , and client apis 
 for retrieving and interacting with data . at the heart of the 
 panorama module
is a comprehensive database 
 schema that closely matches the skyline document data model for capturing 
 the variety of information contained in skyline files .
a web - browser 
 interface ( figure 2 ) lets the user explore 
 and interrogate results contained in uploaded skyline documents , including 
 chromatogram views , peak area graphs , and an interactive viewer for 
 ms / ms spectra ( supporting information note 1 ) .
users can upload skyline documents to a hierarchical folder structure 
 and view the precursors and transitions measured in a skyline document 
 ( supporting information figure 1 ) .
more 
 detailed views are available for the proteins ( supporting information figure 2 ) and peptides ( supporting information figure 3 ) contained in 
 each document .
an enhanced search interface allows searching for a 
 protein , peptide , or specific modifications contained in the documents 
 published to a folder and , optionally , its subfolders ( supporting information figure 4 ) .
panorama web - browser 
 interface showing the precursor list in a 
 skyline document . clicking on one of the peptide sequences
provides 
 a more detailed view for the peptide that includes chromatograms for 
 the precursors in all of the replicates .
a graph showing the peak 
 areas for the peptide measured in individual replicates is also displayed . 

peak areas can be grouped by a replicate annotation , as shown above 
 for the annotation  condition  ( disease and healthy ) . 

the source document can be downloaded via a download link for viewing 
 in skyline .
labkey server , developed 
 over the past decade primarily by the 
 programmers at labkey software , is a data management platform that 
 provides the ability to organize data into a customizable workspace 
 with a hierarchical folder structure .
data can be separated into folders 
 by organism , biological matrix , ms instrumentation , and so on , which 
 allows users to identify and organize experiments over a period of 
 time .
panorama users can control access to the data in each folder 
 by configuring the permission settings to keep the data completely 
 private or can provide access to select collaborators or journal reviewers 
 prior to publication ( supporting information note 
 1 ) . upon publication , or whenever the data owners choose , making 
 a simple change to the security settings can make the data available 
 to the general public .
the comprehensive access control model implemented 
 in labkey server makes panorama a viable option for individual laboratories , 
 companies , and other organizations that want secure and flexible management 
 of their accumulated targeted assay data .
data in a panorama 
 database can be accessed through client libraries available in several 
 programming languages , including r , javascript , java , perl , and python . 

programmatic access to the data facilitates building external applications 
 for data analysis and designing custom user interfaces
. selected reaction 
 monitoring ( srm ) has been recognized as a method for detecting and 
 quantifying proteins of interest in a given sample with high sensitivity 
 and specificity .
a key requirement in 
 developing an srm assay is selecting peptides that uniquely identify 
 a protein and have physicochemical properties that make them readily 
 detectable in a mass spectrometer .
also important is the selection 
 of candidate fragment ions for each of the peptides of interest .
current 
 approaches use a combination of spectral libraries from shotgun experiments and de novo prediction algorithms or broad measurement and subsequent method refinement to guide the selection of peptides .
the former 
 assumes that peptides most frequently observed in shotgun experiments 
 will produce a good response in a targeted proteomics setting , and 
 the latter necessitates concerted effort for each new method developed . 

although spectrum libraries can accurately predict which fragment 
 ions will produce the most intense signal on a triple quadrupole instrument , these libraries do not always have adequate 
 coverage of the entire proteome , especially low - abundance proteins 
 such as human transcription factors and kinases .
even the fragment ion measurements in these libraries may 
 differ from targeted measurements due to instrument differences or 
 interference that goes undetected in spectrum measurement . to overcome 
 these limitations , assay development efforts may measure all precursor 
 and product ions for proteins of interest , under experimental conditions 
 with safeguards like reference standard peptides , and then iteratively 
 refine assays . once an assay has been established , 
 it can be used to guide the selection of peptides and product ions 
 to measure in any new experiment under similar conditions .
this benefit , 
 however , is frequently lost or cumbersome without organized storage 
 and recall .
panorama solves this problem by providing the option 
 of organizing assay results in special chromatogram library folders 
 ( supporting information note 2 ) .
assays 
 that have been curated in skyline can be published to designated library 
 folders in panorama , where they may be subsequently downloaded as 
 a chromatogram library file .
the library download is updated with 
 new assays each time a skyline document is published to the library 
 folder .
users can create library folders for each biological source , 
 sample matrix , experimental protocol , and mass spectrometer type used 
 in any experiment .
this makes these libraries as specific to a given 
 experimental setup as necessary and the assays reproducible under 
 similar settings .
chromatogram libraries can be downloaded for use 
 in skyline to pick optimal peptides for proteins and optimal product 
 ions for precursors in a new experiment .
chromatogram libraries also 
 contain representative chromatograms and peak areas for each precursor 
 that can be compared with newly acquired data for validation of peptide 
 identification ( supporting information figure 
 5 and supporting information note 2 ) .
skyline 
 captures extensive information about the targeted proteomics methods 
 it is used to design as well as the acquired results imported and 
 curated within its user interface .
this information is stored in the 
 skyline document , a human readable xml format file , and a companion 
 binary data file .
the relational database schema in which panorama 
 stores data was designed to mirror closely the skyline data model 
 and to include the full information content it contains , such as complex 
 peptide modification information , normalized retention times ( irt ) , optimized collision energy , and declustering potential .
panorama is fully integrated 
 into the skyline user interface , allowing users to configure one or 
 more panorama servers in skyline and to publish documents to a folder 
 within a server of choice through a point - and - click user interface 
 ( supporting information figure 6 and supporting 
 information note 1 ) . all documents published to panorama may 
 be downloaded again through a web - browser link and reviewed within 
 skyline .
documents published to designated library folders in panorama 
 can be combined into library files and used in skyline to pick optimal 
 peptides and product ions , as well as to compare acquired data to 
 results contained in panorama for peak identification confidence . 

panorama development keeps pace with skyline development to include 
 new features added to skyline such as replicate annotations and mass 
 errors in chromatograms extracted from high - resolution mass spectra .
panorama can 
 be installed on a server in a lab by following one of the installation 
 options available for labkey server .
the latest release of labkey 
 server , with the panorama module , can be downloaded from http://labkey.com/download-labkey-server , and commercial support is available to assist server setup . for 
 laboratories without the resources , expertise , or interest to maintain 
 their own panorama server , a public server , panoramaweb ,
is hosted 
 at the university of washington by the maccoss lab ( https://panoramaweb.org ) .
anyone can request a free project in which they will have full 
 administrative rights .
these rights includes the ability to configure 
 the project workspace with the desired folder hierarchy , project membership , 
 and access control policies , as one would on any private installation . 

panoramaweb currently hosts projects for 66 different laboratories , 
 and a total of 779 skyline documents have been uploaded to the server . 

latest figures on panoramaweb usage can be viewed at the following 
 link : https://panoramaweb.org/labkey/wiki/home/page.view?name=dashboard .
in addition to the ability to configure 
 access permissions to suit the needs of a project , panorama leverages 
 a variety of components and features available in labkey server to 
 provide an environment that facilitates collaboration and data sharing . 

this includes the availability of client libraries in several programming 
 languages that allow external applications to programmatically access 
 the data in a panorama database .
these libraries can also be used 
 on the panorama server to design custom interfaces .
the comprehensive 
 panorama database schema is exposed to the user to enable creating 
 custom views of the data or even writing custom sql scripts .
data 
 analysis can be performed within panorama using familiar tools such 
 as the r software environment for statistical computing .
panoramaweb 
 ( https://panoramaweb.org ) , the public 
 panorama server maintained by the department of genome sciences at 
 the university of washington , hosts several multilab projects that 
 showcase the suitability of the platform as a solution for collaborative 
 targeted proteomics efforts .
the clinical proteomic tumor analysis 
 consortium ( cptac ) assay 
 portal ( https://assays.cancer.gov/ ) , designed to disseminate high - quality proteomics assays , 
 is tied to the cptac assay portal project ( https://daily.panoramaweb.org/labkey/cptac_assays.url ) on panoramaweb .
cptac members have published detailed assay characterization 
 data , contained in skyline documents , to this project .
the cptac - hosted 
 web portal uses the labkey server programmatic interfaces to request 
 data from the panoramaweb database .
several analysis scripts , written 
 in the r programming language , were developed by cptac members to 
 ensure data quality and to evaluate assay performance .
these scripts 
 were integrated into their panoramaweb project ( supporting information figure 7 ) , and the results from running 
 these scripts are displayed in the cptac - hosted portal .
as new assays 
 are submitted , these scripts are executed on panormaweb via programmatic 
 requests , and the results are imported into the assay portal database .
panoramaweb was chosen as the repository for the american biomedical 
 research foundation ( abrf ) standards proteomics research group ( sprg ) 
 2013 study in which over 40 anonymized laboratories published their 
 results to a panoramaweb project .
panorama also has the ability to 
 import excel spreadsheets , allowing results from participants that 
 did not use skyline for data processing to be added to the same project 
 ( supporting information figure 8) .
in addition , 
 a custom summary table was uploaded to panoramaweb ( supporting information figure 9 ) , enabling results from all 
 of the participants to be captured in one overview table with hyperlinks 
 leading to the results for individual participants .
results from analysis 
 scripts , developed by the sprg members to assess data quality , were 
 added to each participant s folder on panoramaweb ( supporting information figure 10 ) .
the project 
 benefitted from the data management and organization features of panorama , 
 and data analysis was greatly simplified by the ability to easily 
 access results from multiple participating laboratories .
processed 
 data as well as analysis scripts and their results are available in 
 the abrf sprg project on panoramaweb ( https://panoramaweb.org/labkey/abrf-sprg-2013.url ) .
panoramaweb also hosts targeted phosphoproteomics data from 
 an 
 assay developed under the auspices of the library of integrated network - based 
 cellular signatures ( lincs ) project ( www.lincsproject.org ) .
the lincs consortium engages several centers that generate molecular 
 phenotypic data for cellular perturbations using different assay technologies . 

it is extremely important that data provenance , traceability , and 
 interoperability are maintained and available to consortium members 
 and other consumers of the data .
this particular 
 use case illustrates the flexibility and system 
 interoperability that panoramaweb creates .
execution of a multicenter 
 consortium project requires rigorous attention to sample and assay 
 annotation with appropriate metadata to promote reuse of the data 
 that is generated .
the skyline document model allows faithful assignment 
 of dozens of pieces of metadata to hundreds of samples in individual 
 studies . after publishing lincs data to panoramaweb at https://daily.panoramaweb.org/labkey/lincs.url ,
the combination of quantitative ms data with sample metadata annotations 
 allow automated creation of data formats that can then be leveraged 
 by other components of the consortium . using the r programming language 
 interface
, panorama enables complete data processing from a skyline 
 document , including normalization , outlier removal , probe failure 
 detection , and summarization , to exporting results in the gene cluster 
 text ( gct ) file format ( supporting information 
 figure 11 ) , a recognized standard for analysis of gene expression 
 data .
this allows compatibility with a number of other downstream 
 tools such as the gene - e visualization and analysis platform ( http://www.broadinstitute.org/cancer/software/gene-e/index.html ) and allows direct import into the lincs connectivity map portal 
 ( http://www.lincscloud.org ) .
panorama addresses the need to archive and share an increasing 
 number of documents being generated by the growing adoption of skyline 
 in targeted proteomics workflows . over time
, experiments stored in 
 a panorama server can serve as a knowledge base of curated targeted 
 assays for the proteomics research community .
development of panorama 
 has been a collaborative process to enable the implementation of components 
 and features useful to end users and to identify and explore its potential 
 beyond a data archival system .
we have described several multilab 
 collaborative projects hosted on panoramaweb that are using panorama 
 not only for storing and sharing skyline documents but also for analyzing 
 large data sets and presenting results to the broader community .
 panorama 
 is a web application for storing , sharing , analyzing , 
 and reusing targeted assays created and refined with skyline,1 an increasingly popular windows client software 
 tool for targeted proteomics experiments . 
 panorama allows laboratories 
 to store and organize curated results contained in skyline documents 
 with fine - grained permissions , which facilitates distributed collaboration 
 and secure sharing of published and unpublished data via a web - browser 
 interface . 
 it is fully integrated with the skyline workflow and supports 
 publishing a document directly to a panorama server from the skyline 
 user interface . 
 panorama captures the complete skyline document information 
 content in a relational database schema . 
 curated results published 
 to panorama can be aggregated and exported as chromatogram libraries . 
 
 these libraries can be used in skyline to pick optimal targets in 
 new experiments and to validate peak identification of target peptides . 
 
 panorama is open - source and freely available . 
 it is distributed as 
 part of labkey server,2 an open source 
 biomedical research data management system . 
 laboratories and organizations 
 can set up panorama locally by downloading and installing the software 
 on their own servers . 
 they can also request freely hosted projects 
 on https://panoramaweb.org , a panorama server maintained 
 by the department of genome sciences at the university of washington . 
Introduction
Design and Implementation
Use Cases and Public Implementations That Enable
Collaboration
Conclusions