Answers.Body: <p>For (3), <a href="http://www.geneinfinity.org/sp/sp_motif.html" rel="nofollow noreferrer">this page</a> has a lot of links to pattern/motif finding tools. Following through the <a href="http://bio.cs.washington.edu/YMF/YMFWeb/YMFInput.pl" rel="nofollow noreferrer">YMF</a> link on that page, I came across the University of Washington <a href="http://bio.cs.washington.edu/software/motif_discovery#Motif%20Discovery" rel="nofollow noreferrer">Motif Discovery</a> section. Of these <a href="http://www1.cse.wustl.edu/~jbuhler/pgt/" rel="nofollow noreferrer">projection</a> seemed to be the only downloadable tool. I find it interesting how old all these tools are; maybe the introduction of microarrays and NGS has made them all redundant.</p>
  
  <p>Your sub-problem (2) seems similar to the problem I'm having with <em>Nippostrongylus brasiliensis</em> genome sequences, where I'd like to find regions of very high homology (length 500bp to 20kb or more, 95-99% similar) that are repeated throughout the genome. These sequences are killing the assembly.</p>
  
  <p>The main way I can find these regions is by looking at a coverage plot of long nanopore reads mapped to the assembled genome (using GraphMap or BWA). Any regions with substantially higher than median coverage are likely to be shared repeats.</p>
  
  <p>I've played around in the past with chopping up the reads to smaller sizes, which works better for hitting smaller repeated regions that are such a small proportion of most reads that they are never mapped to all the repeated locations. I wrote <a href="https://github.com/gringer/bioinfscripts/blob/master/normalise_seqlengths.pl" rel="nofollow noreferrer">my own script</a> a while back to chop up reads (for a different purpose), which produces a FASTA/FASTQ file where all reads are exactly the same length. For some unknown reason I took the time to document that script "properly" using POD, so here's a short summary:</p>
  
  <blockquote>
    <p>Converts all sequences in the input FASTA file to the same length.
       Sequences shorter than the target length are dropped, and sequences longer
       than the target length are split into overlapping subsequences covering
       the entire range. This prepares the sequences for use in an
       overlap-consensus assembler requiring constant-length sequences (such as
       edena).</p>
  </blockquote>
  
  <p>And here's the syntax:</p>
  
  <pre><code>$ ./normalise_seqlengths.pl -h
  Usage:
      ./normalise_seqlengths.pl &lt;reads.fa&gt; [options]
  
    Options:
      -help
        Only display this help message
  
      -fraglength
        Target fragment length (in base-pairs, default 2000)
  
      -overlap
        Minimum overlap length (in base-pairs, default 200)
  
      -short
        Keep short sequences (shorter than fraglength)
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-09T02:19:01.120Z
Answers.LastActivityDate: 2017-06-09T04:14:39.550Z
Answers.LastEditDate: 2017-06-09T04:14:39.550Z
Answers.UserDisplayName: gringer
Answers.Body: <p>The <a href="http://meme-suite.org" rel="nofollow noreferrer">MEME Suite</a> web site contains a collection of tools for motif analysis (I'm one of the maintainers). It contains two de novo motif discovery tools: <a href="http://meme-suite.org/tools/meme" rel="nofollow noreferrer">MEME</a> and <a href="http://meme-suite.org/tools/dreme" rel="nofollow noreferrer">DREME</a>. Public web applications are provided, but you can also download and build command line tools for a local installation.</p>
  
  <p>For your first goal you could use <a href="http://meme-suite.org/tools/meme" rel="nofollow noreferrer">MEME</a> and select the "Any number of repetitions model" (ANR). For your second goal, you'd use MEME with the "Zero or One Occurrences Per Sequence" (ZOOPS) model. For your third goal you could use FIMO (Find Individual Motif Occurrences), and one or more of the motif databases provided on the <a href="http://meme-suite.org/doc/download.html?man_type=web" rel="nofollow noreferrer">software and database download page</a>.</p>
  
  <p>It sounds like your sequence data is about 30Mb. The MEME web application is limited to 60kb of sequence data, so you'd have to install a local copy of the MEME Suite. MEME would take a long time to analyze a 30Mb sequence database unless you have MPI configured, and lots of cores available. You might want to consider analyzing a randomly selected subset of your sequences. The running time of MEME grows as the cube of the number of sequences. </p>
  
  <p>For short motifs, you may want to use <a href="http://meme-suite.org/tools/dreme" rel="nofollow noreferrer">DREME</a> rather than MEME. DREME is better than MEME at identifying short motifs, but is limited to motifs &lt;= 8 positions wide.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-13T19:28:19.813Z
Answers.LastActivityDate: 2017-06-13T22:20:11.820Z
Answers.LastEditDate: 2017-06-13T22:20:11.820Z
Answers.UserDisplayName: Charles E. Grant
Answers.Body: <p>Check out <a href="http://homer.ucsd.edu/homer/" rel="nofollow noreferrer">HOMER</a>. "Software for motif discovery and next generation sequencing analysis", it's what my lab uses currently for finding eRNA motifs.</p>
  
  <p>Edit: For @ShanZhengYang "HOMER was designed as a de novo motif discovery algorithm..." 
  <a href="http://homer.ucsd.edu/homer/introduction/basics.html" rel="nofollow noreferrer">HOMER De Novo Motif</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-12T16:41:09.150Z
Answers.Text: Can one do de novo motif discovery? If not, I'm not entirely sure how I would create a background with this model.
Answers.UserDisplayName: ShanZhengYang
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-12T16:48:24.393Z
Answers.Text: @ShanZhengYang See my edit.
Answers.UserDisplayName: EMiller
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-12T17:11:42.280Z
Answers.Text: Thanks. I'm still not sure given my set-up how to choose a background for Homer however...
Answers.UserDisplayName: ShanZhengYang
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-12T17:41:08.230Z
Answers.Text: I've never used it for De Novo, so take this with a grain of salt, but try the defaults first. Perhaps shoot the developers an email.
Answers.UserDisplayName: EMiller
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-13T19:19:43.607Z
Answers.Text: To expand on @EMiller 's comment, for many model organisms, HOMER can figure out a background model for you. You just have to install the supporting files for your model organism using  configureHomer.pl
Answers.UserDisplayName: Charles E. Grant
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-12T16:02:21.920Z
Answers.LastActivityDate: 2017-06-12T16:48:00.760Z
Answers.LastEditDate: 2017-06-12T16:48:00.760Z
Answers.UserDisplayName: EMiller
Answers.Body: <p>it is under development, but maybe <a href="https://github.com/soedinglab/BaMMmotif" rel="nofollow noreferrer">BaMMmotif!</a> is something for you? Its main selling point is that it can look for motifs enriched in a set of sequences of equal length <em>de novo</em>. If you can't/don't want to supply a negative set it learns one from the positive sequences. There is a wealth of options to choose from if you have more information about your sequences: there are different models for "zero or one", "one" and "multiple" occurrences of the motif.</p>
  
  <p>You can also use it to look for known motifs, if you encode them as an XXmotif PWM. If you have a file with motifs (like binding sites) you can use that as initialization as well.</p>
  
  <p>While I have not used the software myself, the authors are very responsive on git and the installation instructions seem pretty straightforward.</p>
  
  <p>EDIT: Apparently the software was developed with ChIP experiments as the usual use case scenario, it might misbehave for larger sequences or take super long to run.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-09T15:39:34.597Z
Answers.Text: Hi galicae, thanks for your answer and welcome to Bioinformatics Stack Exchange. Your answer is a bit short, and could do with a bit more explanation. Could you please add a bit more information into your answer about BaMMmotif? Is it a tool that you have used before? Would this be an appropriate program to find novel shared motifs, repeated motifs within the same sequence, or known motifs?
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-12T06:27:09.777Z
Answers.Text: @gringer fair enough, edited.
Answers.UserDisplayName: galicae
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-09T10:20:12.173Z
Answers.LastActivityDate: 2017-06-12T09:44:17.427Z
Answers.LastEditDate: 2017-06-12T09:44:17.427Z
Answers.UserDisplayName: galicae
Answers.Body: <p>Most tools I know of looks for enrichment of specific motifs - but that requires that you have a set of sequences which are of special interest and a background set to test against.</p>
  
  <p>Is that your case?</p>
  
  <p>Update after comments 12th June 2017.</p>
  
  <p>You could try the <a href="http://meme-suite.org/" rel="nofollow noreferrer">meme suite</a> more specifically <a href="http://meme-suite.org/tools/meme" rel="nofollow noreferrer">the motif finder</a> </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-09T15:35:48.947Z
Answers.Text: Hi Kristoffer Vitting-Seerup, thanks for your comment and welcome to Bioinformatics Stack Exchange. We appreciate any comments like this that help people ask better questions. I have flagged your response as "not an answer" because I think it should be added as a comment to the question (which you can't do yourself until you reach a reputation of 50).
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-09T18:59:28.930Z
Answers.Text: Hi there, yes this would be a better comment to the original question. This is the SO style. Regarding your question, I don't have a background set
Answers.UserDisplayName: ShanZhengYang
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-09T09:41:25.517Z
Answers.LastActivityDate: 2017-06-13T16:21:27.300Z
Answers.LastEditDate: 2017-06-13T16:21:27.300Z
Answers.UserDisplayName: Kristoffer Vitting-Seerup
Body: <p>I have around ~3,000 short sequences of approximately ~10Kb long. What are the best ways to find the motifs among all of these sequences? Is there a certain software/method recommended?</p>
  
  <p>There are several ways to do this. My goal would be to:</p>
  
  <p>(1) Check for motifs repeated within individual sequences</p>
  
  <p>(2) Check for motifs shared among all sequences</p>
  
  <p>(3) Check for the presence of "expected" or known motifs</p>
  
  <p>With respect to #3, I'm also curious if I find e.g. trinucleotide sequences, how does one check the context around these regions?</p>
  
  <p>Thank you for the recommendations/help!</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-08T20:34:44.747Z
Comments.Text: Are you looking for motifs shared by all the sequences? Motifs repeated in each sequence? Are you looking for the presence of specific, known motifs?
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-08T21:23:33.137Z
Comments.Text: "Are you looking for motifs shared by all the sequences? Motifs repeated in each sequence?" I was looking for both motifs shared by all sequences and repeated in sequences, but actually it would be interesting to check each sequence for a repeated sequence. "Are you looking for the presence of specific, known motifs?" No, but it would be interesting to look into this in retrospect
Comments.UserDisplayName: ShanZhengYang
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-08T21:26:40.827Z
Comments.Text: OK, please [edit] your question and ass this information. Comments are easy to miss, hard to read and can be deleted without warning. The three problems (de-novo shared motif identification, de-novo repeated motif identification and detecting known motifs) are different and each requires its own approach.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-08T21:34:34.133Z
Comments.Text: @terdon Of course
Comments.UserDisplayName: ShanZhengYang
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-08T23:06:53.107Z
Comments.Text: Do you mean 10kb in total, or 3000 sequences, each of length around 10kb?
Comments.UserDisplayName: Charles E. Grant
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-12T06:53:03.700Z
Comments.Text: @CharlesE.Grant 3000 sequences, each of length around 10kb
Comments.UserDisplayName: ShanZhengYang
Tags: <motifs><software-recommendation>
Title: What motif finding software is available for multiple sequences ~10Kb?
ThreadText: # What motif finding software is available for multiple sequences ~10Kb?
  
  (Asked by: username_0 on 2017-06-08)
  
  I have around ~3,000 short sequences of approximately ~10Kb long. What are the best ways to find the motifs among all of these sequences? Is there a certain software/method recommended?
  
  There are several ways to do this. My goal would be to:
  
  (1) Check for motifs repeated within individual sequences
  
  (2) Check for motifs shared among all sequences
  
  (3) Check for the presence of "expected" or known motifs
  
  With respect to #3, I'm also curious if I find e.g. trinucleotide sequences, how does one check the context around these regions?
  
  Thank you for the recommendations/help!
  
  **Tags:** motifs, software-recommendation
  
  > **Comments:**
  > **username_1**: Are you looking for motifs shared by all the sequences? Motifs repeated in each sequence? Are you looking for the presence of specific, known motifs?  
  > **username_0**: "Are you looking for motifs shared by all the sequences? Motifs repeated in each sequence?" I was looking for both motifs shared by all sequences and repeated in sequences, but actually it would be interesting to check each sequence for a repeated sequence. "Are you looking for the presence of specific, known motifs?" No, but it would be interesting to look into this in retrospect  
  > **username_1**: OK, please [edit] your question and ass this information. Comments are easy to miss, hard to read and can be deleted without warning. The three problems (de-novo shared motif identification, de-novo repeated motif identification and detecting known motifs) are different and each requires its own approach.  
  > **username_0**: @username_1 Of course  
  > **username_2**: Do you mean 10kb in total, or 3000 sequences, each of length around 10kb?  
  > **username_0**: @CharlesE.username_2 3000 sequences, each of length around 10kb  
  
  ---
  
  ## 5 answers
  
  ### Answer by username_3 on 2017-06-09. Score: 1 | Accepted answer
  
  For (3), [this page](http://www.geneinfinity.org/sp/sp_motif.html) has a lot of links to pattern/motif finding tools. Following through the [YMF](http://bio.cs.washington.edu/YMF/YMFWeb/YMFInput.pl) link on that page, I came across the University of Washington [Motif Discovery](http://bio.cs.washington.edu/software/motif_discovery#Motif%20Discovery) section. Of these [projection](http://www1.cse.wustl.edu/~jbuhler/pgt/) seemed to be the only downloadable tool. I find it interesting how old all these tools are; maybe the introduction of microarrays and NGS has made them all redundant.
  
  Your sub-problem (2) seems similar to the problem I'm having with *Nippostrongylus brasiliensis* genome sequences, where I'd like to find regions of very high homology (length 500bp to 20kb or more, 95-99% similar) that are repeated throughout the genome. These sequences are killing the assembly.
  
  The main way I can find these regions is by looking at a coverage plot of long nanopore reads mapped to the assembled genome (using GraphMap or BWA). Any regions with substantially higher than median coverage are likely to be shared repeats.
  
  I've played around in the past with chopping up the reads to smaller sizes, which works better for hitting smaller repeated regions that are such a small proportion of most reads that they are never mapped to all the repeated locations. I wrote [my own script](https://github.com/username_3/bioinfscripts/blob/master/normalise_seqlengths.pl) a while back to chop up reads (for a different purpose), which produces a FASTA/FASTQ file where all reads are exactly the same length. For some unknown reason I took the time to document that script "properly" using POD, so here's a short summary:
  
  > Converts all sequences in the input FASTA file to the same length.
  > Sequences shorter than the target length are dropped, and sequences longer
  > than the target length are split into overlapping subsequences covering
  > the entire range. This prepares the sequences for use in an
  > overlap-consensus assembler requiring constant-length sequences (such as
  > edena).
  
  And here's the syntax:
  
  ```
  $ ./normalise_seqlengths.pl -h
  Usage:
      ./normalise_seqlengths.pl <reads.fa> [options]
  
    Options:
      -help
        Only display this help message
  
      -fraglength
        Target fragment length (in base-pairs, default 2000)
  
      -overlap
        Minimum overlap length (in base-pairs, default 200)
  
      -short
        Keep short sequences (shorter than fraglength)
  
  ```
  
  
  ---
  
  ### Answer by username_2 on 2017-06-13. Score: 3
  
  The [MEME Suite](http://meme-suite.org) web site contains a collection of tools for motif analysis (I'm one of the maintainers). It contains two de novo motif discovery tools: [MEME](http://meme-suite.org/tools/meme) and [DREME](http://meme-suite.org/tools/dreme). Public web applications are provided, but you can also download and build command line tools for a local installation.
  
  For your first goal you could use [MEME](http://meme-suite.org/tools/meme) and select the "Any number of repetitions model" (ANR). For your second goal, you'd use MEME with the "Zero or One Occurrences Per Sequence" (ZOOPS) model. For your third goal you could use FIMO (Find Individual Motif Occurrences), and one or more of the motif databases provided on the [software and database download page](http://meme-suite.org/doc/download.html?man_type=web).
  
  It sounds like your sequence data is about 30Mb. The MEME web application is limited to 60kb of sequence data, so you'd have to install a local copy of the MEME Suite. MEME would take a long time to analyze a 30Mb sequence database unless you have MPI configured, and lots of cores available. You might want to consider analyzing a randomly selected subset of your sequences. The running time of MEME grows as the cube of the number of sequences.
  
  For short motifs, you may want to use [DREME](http://meme-suite.org/tools/dreme) rather than MEME. DREME is better than MEME at identifying short motifs, but is limited to motifs <= 8 positions wide.
  
  
  ---
  
  ### Answer by username_4 on 2017-06-12. Score: 2
  
  Check out [HOMER](http://homer.ucsd.edu/homer/). "Software for motif discovery and next generation sequencing analysis", it's what my lab uses currently for finding eRNA motifs.
  
  Edit: For @username_0 "HOMER was designed as a de novo motif discovery algorithm..."
  [HOMER De Novo Motif](http://homer.ucsd.edu/homer/introduction/basics.html)
  
  > **Comments:**
  > **username_0**: Can one do de novo motif discovery? If not, I'm not entirely sure how I would create a background with this model.  
  > **username_4**: @username_0 See my edit.  
  > **username_0**: Thanks. I'm still not sure given my set-up how to choose a background for Homer however...  
  > **username_4**: I've never used it for De Novo, so take this with a grain of salt, but try the defaults first. Perhaps shoot the developers an email.  
  > **username_2**: To expand on @username_4 's comment, for many model organisms, HOMER can figure out a background model for you. You just have to install the supporting files for your model organism using  configureHomer.pl  
  
  ---
  
  ### Answer by username_5 on 2017-06-09. Score: 1
  
  it is under development, but maybe [BaMMmotif!](https://github.com/soedinglab/BaMMmotif) is something for you? Its main selling point is that it can look for motifs enriched in a set of sequences of equal length *de novo*. If you can't/don't want to supply a negative set it learns one from the positive sequences. There is a wealth of options to choose from if you have more information about your sequences: there are different models for "zero or one", "one" and "multiple" occurrences of the motif.
  
  You can also use it to look for known motifs, if you encode them as an XXmotif PWM. If you have a file with motifs (like binding sites) you can use that as initialization as well.
  
  While I have not used the software myself, the authors are very responsive on git and the installation instructions seem pretty straightforward.
  
  EDIT: Apparently the software was developed with ChIP experiments as the usual use case scenario, it might misbehave for larger sequences or take super long to run.
  
  > **Comments:**
  > **username_3**: Hi username_5, thanks for your answer and welcome to Bioinformatics Stack Exchange. Your answer is a bit short, and could do with a bit more explanation. Could you please add a bit more information into your answer about BaMMmotif? Is it a tool that you have used before? Would this be an appropriate program to find novel shared motifs, repeated motifs within the same sequence, or known motifs?  
  > **username_5**: @username_3 fair enough, edited.  
  
  ---
  
  ### Answer by username_6 on 2017-06-09. Score: 1
  
  Most tools I know of looks for enrichment of specific motifs - but that requires that you have a set of sequences which are of special interest and a background set to test against.
  
  Is that your case?
  
  Update after comments 12th June 2017.
  
  You could try the [meme suite](http://meme-suite.org/) more specifically [the motif finder](http://meme-suite.org/tools/meme)
  
  > **Comments:**
  > **username_3**: Hi username_6 username_6, thanks for your comment and welcome to Bioinformatics Stack Exchange. We appreciate any comments like this that help people ask better questions. I have flagged your response as "not an answer" because I think it should be added as a comment to the question (which you can't do yourself until you reach a reputation of 50).  
  > **username_0**: Hi there, yes this would be a better comment to the original question. This is the SO style. Regarding your question, I don't have a background set  
  
  ---
  
  