I started working on my first project with PacBio sequencing data, and after 2 days filled with fruitless googling, missing libraries and failed c compilations, I decided it's time to ask for help.

The information on tools and pipelines for PacBio data is scattered everywhere and most of it seems horribly out of date. I was hoping some of you could help me to get started and other lost souls in the future will hopefully find this post and save some time and frustration.

**The data:**

I have a set of Primary Analysis Data available, as explained in the first paragraph [here][1]. 

**The tools:**

After some struggle, I managed to compile the latest version of [blasr][2] and [pbalign][3] on Ubuntu 16.04 using pitchfork. I also managed to install the R packages [h5r][4], [pbh5][5], and [seqPatch][6]. If anyone is reading this and has trouble with R and HDF5 libraries under Ubuntu, see my question [here][7].

**What I don't have:**

I don't have access to a server with the SMRT Link platform.

**What I would like to do:**

I would like to access the interpulse duration (IPD), as explained in this [white paper][8], and preferably access this data in R. I am open to suggestions for other tools/programming languages as well though.

**Problem:**

I need cmp.h5 files to load the IPDs from the R packages. How do I generate those? When I try to run pbalign, it says `pbalign no longer supports CMP.H5 Output in 3.0`. Is there any other way to get to the IPDs without going through cmp.h5 files?

Thank you very much for your time.

(Edit note: I realized my questions were too broad and specified a single problem to start with instead.)

  [1]: https://github.com/PacificBiosciences/SMRT-Analysis/wiki/Data-files-you-received-from-your-service-provider
  [2]: https://github.com/PacificBiosciences/blasr
  [3]: https://github.com/PacificBiosciences/pbalign
  [4]: http://r-forge.r-project.org/projects/h5r/
  [5]: https://github.com/PacificBiosciences/R-pbh5
  [6]: https://github.com/zhixingfeng/seqPatch
  [7]: https://askubuntu.com/questions/1010676/r-package-h5r-configure-error-cant-find-hdf5
  [8]: https://www.pacb.com/wp-content/uploads/2015/09/WP_Detecting_DNA_Base_Modifications_Using_SMRT_Sequencing.pdf
 - What organism is your data from?
 - What is the genome size of the organism? What Coverage do you have?
 - What PacBio instrument was your data generated on RSII (output is a bax.h5 file) or Sequel (output is an unaligned .bam file)?

The instrument that the data was generated on is important as it will determine the BFX tools you can use for analysis.

 - **SMRT Portal** is designed to be used with RSII data and accept raw bax.h5 files for input. It was last updated in Nov 2014 so unfortunately not actively maintained. You will be reserved to scrolling through outdated GitHub info and the interwebs for analysis help.

 - **SMRT Link** is designed to be used with Sequel data and accept an unaligned .bam file for input. It is somewhat backwards compatible with RSII data. There is a bax2bam command that allows you to convert RSII data to the same unaligned bam format of Sequel data. This is sufficient for most applications but I don't think it works (correct me if I am wrong?) for base mod work because the IDP info is not conserved upon file conversion.

I am not aware of a way you can get around using a cmp.h5 file for IDP information. It is also likely it will be hard to get out of downloading and using either SMRT Portal of SMRT Link in one way or another. Luckily both of them can be used relatively easily on a workstation (dependent on genome size of your organism). SMRT Portal can be run in GUI format on a workstation and SMRT Link can be run in command-line only format (without having to set up a full SMRT Link server). 

**SMRT Portal**

 - [Here is the download for SMRT Portal][1] go to the bottom and click the link "Previous release of SMRT Analysis for RSII"
 - [Here is a SMRT Portal Help Page][2] this can help you use the GUI
 - [Here is a Base Mod Technical Note][3] **You should look at this walkthrough regardless if you use SRMT Portal or SMRT Link as the output files are similar and so is the process**

**SMRT Link**

 - [Here is the][4] download for SMRT Link
 - [Here is][5] official instructions for downloading SMRT Link see page 8 for the command line only tools.
 - [Here is a Biostars link to instructions on how to install the command-line only tools][6] It is wayyyy better than the official instructions.
 - [Here is the SMRT Tools Reference Guide ][7]- It is an in-depth list of all the commands possible in smrt link and their options. Check out pages 41 -43 for MotifMaker. 

 - Check out pages 52-58 of the reference guide for pbsmrtpipe, this will allow you to run the whole ds_motif_modification_analysis pipeline that allows you to generate the .csv file seen in the whitepaper. Specifically page 58 shows the command that would run this pipeline (with a slight ID modification).

Once you have ran a base mod pipeline in either SMRT Portal or SMRT Link (via pbsmrtpipe) you should have output .csv, .gff. and cmp.h5 files that you can do tertiary analysis on using whatever you want. There are also a few tools available from PacBio that run downstream of the initial analysis.

 - [PacBio Base Mod tools][8] this is the link to their GitHub of additional tools. It looks like only kineticsTools, MotifMaker, and MotifFinder have been updated for Sequel data. 

There are also a handful of methods developed by other researchers to use IDP / base mod data from PacBio data. You could look at some of the published papers to get additional ideas.

 - [Many papers that utilize this type of IDP work are on PacBio's Website][9] look at the publication section at the bottom of the page. 
 - [Metagenomic binning and association of plasmids with bacterial host genomes using DNA methylation.][10] This one would be cool to see what bfx pipeline they did. It is from Mt Sinai and SEMA4. Unfortunately it is behind a paywall. 
 - [AgIn: measuring the landscape of CpG methylation of individual repetitive elements][11] Agln is unique because used the IDP data to map regional methylation (CpG) islands in eukaryotic organisms (large genomes) at only 20-40 fold coverage of PacBio data.  


  [1]: https://www.pacb.com/support/software-downloads/
  [2]: http://files.pacb.com/software/smrtanalysis/2.0.0/doc/smrtportal/help/SMRT_Portal.htm
  [3]: https://github.com/PacificBiosciences/Bioinformatics-Training/wiki/Methylome-Analysis-Technical-Note
  [4]: https://www.pacb.com/support/software-downloads/
  [5]: https://go.pardot.com/l/1652/2017-07-10/3ttgnn/1652/187151/SMRT_Link_Installation_v501.pdf
  [6]: https://www.biostars.org/p/273447/
  [7]: https://go.pardot.com/l/1652/2017-07-10/3ttgnq/1652/187153/SMRT_Tools_Reference_Guide.pdf
  [8]: https://github.com/PacificBiosciences/Bioinformatics-Training/wiki/Base-Modification-Tools
  [9]: https://www.pacb.com/applications/epigenetics/
  [10]: https://www.ncbi.nlm.nih.gov/pubmed/29227468
  [11]: https://academic.oup.com/bioinformatics/article/32/19/2911/2196534