Answers.Body: <p>I'm one of the EMBLmyGFF3 developers. If the compressed annotation is 396MB as mentioned by @terdon it is not a small annotation... In the worse case we have tested we never waited more than few minutes. We haven't done any particular speed test. It would be valuable if you could tell us how long it has taken in your case (If you didn't kill the process before the end...).</p>
  
  <p>At least we could try to implement some nice features, like to see that the tool is working , or/and see which percentage of the file has been already processed.</p>
  
  <p>If it is really slow we could think to try to improve its speed.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-10T17:13:59.660Z
Answers.Text: Does EMBLmyGFF3 accept .gz compressed files? Is it better? I have started the transformation at 5 pm today. The size of the uncompressed annotation file is 3.1 GB. The genome file is about 100 MB uncompressed. I'll let you know if/when it finishes. It would definitely be useful to have a progress bar and a sign that the tool is working properly. Thanks a lot!
Answers.UserDisplayName: Biomagician
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-10T17:46:00.470Z
Answers.Text: Still going and using 50 GB of memory (on a 16 GB physical RAM machine) but less than 100% of a CPU. So, it's using the hard disk to store some stuff.
Answers.UserDisplayName: Biomagician
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-12T01:09:17.830Z
Answers.Text: The process has taken about 24 hours in total on 16 GB physical RAM and 8 cores.
Answers.UserDisplayName: Biomagician
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-12T09:36:26.057Z
Answers.Text: Ok it is obviously really slow... Using .gz compressed file will not change anything here. We will investigate how to improve the speedness.
Answers.UserDisplayName: juke34
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-15T10:29:08.550Z
Answers.Text: I get a lot of errors of the type >>antisense_RNA<< is not a valid EMBL feature type. I found http://www.insdc.org/files/feature_table.html#2.3 but I could not locate a table with all the valid EMBL feature types. Do you know where I could find this? I get the error with miRNA as well, would ncRNA be the correct feature type in the EMBL format?
Answers.UserDisplayName: Biomagician
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-16T19:54:01.937Z
Answers.Text: You will find the complete list of features and qualifiers accepted by EBI here: https://www.ebi.ac.uk/ena/WebFeat/
Answers.UserDisplayName: juke34
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-10T16:29:53.673Z
Answers.LastActivityDate: 2018-03-10T16:29:53.673Z
Answers.UserDisplayName: juke34
Answers.Body: <p>That error is telling you that the script you were running was killed by signal <code>9</code>, also known as <code>SIGKILL</code>, while executing the command <code>_seqret "$@"</code>. This is the way the operating system will kill processes that are asking for more memory than it can provide. </p>
  
  <p>Note that removing <code>_secret "$@"</code> is not the solution since that's what's actually doing the job. <code>$@</code> is a bash array that contains all the arguments passed to a bash script. The <code>/Users/me/anaconda/bin/seqret</code> file is just a simple script that sets up a few variables and then launches <code>_seqret</code>, passing it the arguments you gave to <code>/Users/me/anaconda/bin/seqret</code> as <code>$@</code>. So deleting that lines just makes the script useless.</p>
  
  <p>Based on your question, I am guessing you're running macOS on a laptop, so it isn't unreasonable that something that needs to read an entire genome and what looks like a full set of annotations will run out of memory on a laptop. Of course, this is a pretty small genome, but the annotations are still 396MB compressed, so depending on your machine's specs and what kind of internal processing the command is doing, it isn't too surprising that it fails. </p>
  
  <p>So, either run this on a more powerful machine, or try splitting the annotations file into smaller files and running it sequentially on each of those.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-10T00:48:59.633Z
Answers.LastActivityDate: 2018-03-10T14:17:43.210Z
Answers.LastEditDate: 2018-03-10T14:17:43.210Z
Answers.UserDisplayName: terdon
Body: <p>I am using the tool <a href="http://emboss.sourceforge.net/apps/release/6.6/emboss/apps/seqret.html" rel="nofollow noreferrer">seqret</a> from emboss to transform an annotation file in GFF3 format and a FASTA file into an EMBL file because Wormbase does not supply an EMBL file with annotation and sequence together. I am running it on Mac OS X. I got the gff3 and fasta files from <a href="ftp://ftp.wormbase.org/pub/wormbase/releases/current-production-release/species/c_elegans/PRJNA13758/" rel="nofollow noreferrer">Wormbase</a>. I installed emboss using conda from the bioconda channel. I use the following command:</p>
  
  <p><code>seqret -sequence c_elegans.PRJNA13758.WS263.genomic.fa -feature -fformat gff -fopenfile c_elegans.PRJNA13758.WS263.annotations.gff3 -osformat embl -auto -outseq c_elegans.PRJNA13758.WS263.embl</code></p>
  
  <p>It fails with the following error:</p>
  
  <pre><code>/Users/me/anaconda/bin/seqret`: line 7: 54774 Killed: 9               _seqret "$@"
  </code></pre>
  
  <p><code>/Users/me/anaconda/bin/seqret</code> is a short file that indeed contains <code>_seqret "$@"</code> on line 7:</p>
  
  <pre><code>#!/bin/sh
  BIN_DIR=$(dirname $(which seqret))
  export EMBOSS_ACDROOT=$BIN_DIR/../share/EMBOSS/acd/
  export EMBOSS_DATA=$BIN_DIR/../share/EMBOSS/data/
  export PLPLOT_LIB=$BIN_DIR/../share/EMBOSS/
  _seqret "$@"
  </code></pre>
  
  <p>I tried to erase it but that did not help.</p>
  
  <p>I also tried <a href="https://github.com/NBISweden/EMBLmyGFF3" rel="nofollow noreferrer">EMBLmyGFF3</a> in a Python 2 environment:</p>
  
  <p><code>EMBLmyGFF3 c_elegans.PRJNA13758.WS263.annotations.gff3 c_elegans.PRJNA13758.WS263.genomic.fa --topology linear --molecule_type 'genomic DNA' --transl_table 1 --species 'Caenorhabditis elegans' --locus_tag CELE --project_id PRJNA13758 -o c_elegans.PRJNA13758.WS263.annotations.embl</code></p>
  
  <p>but I just had this message on display for hours:</p>
  
  <pre><code>#############################################################################
  # NBIS 2016 - Sweden                                                        #
  # Authors: Martin Norling, Niclas Jareborg, Jacques Dainat                  #
  # Please visit https://github.com/NBISweden/EMBLmyGFF3 for more information #
  #############################################################################
  </code></pre>
  
  <p>Update 11th March:</p>
  
  <p>I started EMBLmyGFF3 yesterday at 5 pm and this morning it started outputting messages such as:</p>
  
  <pre><code>08:22:08 ERROR feature: &gt;&gt;inverted_repeat&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 ERROR feature: &gt;&gt;assembly_component&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 WARNING feature: Unknown qualifier 'genbank' - skipped
  08:22:08 ERROR feature: &gt;&gt;tandem_duplication&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 WARNING feature: Unknown qualifier 'public_name' - skipped
  08:22:08 WARNING feature: Unknown qualifier 'polymorphism' - skipped
  08:22:08 WARNING feature: Unknown qualifier 'other_name' - skipped
  08:22:08 WARNING feature: Unknown qualifier 'consequence' - skipped
  08:22:08 WARNING feature: Unknown qualifier 'variation' - skipped
  08:22:08 ERROR feature: &gt;&gt;biological_region&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 WARNING feature: Unknown qualifier 'balancer_type' - skipped
  08:22:08 WARNING feature: Unknown qualifier 'balancer' - skipped
  08:22:08 ERROR feature: &gt;&gt;protein_match&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 ERROR feature: &gt;&gt;transcript_region&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 ERROR feature: &gt;&gt;transcribed_fragment&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  </code></pre>
  
  <p>...</p>
  
  <pre><code>Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:45:36 ERROR feature: &gt;&gt;piRNA&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:46:00 ERROR feature: &gt;&gt;pseudogenic_tRNA&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:47:59 ERROR feature: &gt;&gt;antisense_RNA&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:47:59 ERROR feature: &gt;&gt;lincRNA&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:48:03 ERROR feature: &gt;&gt;pre_miRNA&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:48:03 ERROR feature: &gt;&gt;miRNA&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:50:37 ERROR feature: &gt;&gt;snRNA&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  09:04:00 ERROR feature: &gt;&gt;miRNA_primary_transcript&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  </code></pre>
  
  <p>For some of these features, it seems OK to not report them. But for others, such as piRNA, I need to find a solution. <em>C. elegans</em> has got 1000s of piRNAs, it would be nice to have them in the EMBL file. I need to find a list of valid EMBL feature types.</p>
  
  <p>Update + ~ 24h computation:</p>
  
  <p>...</p>
  
  <pre><code>Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  12:21:19 ERROR feature: &gt;&gt;scRNA&lt;&lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  Conversion done
  </code></pre>
  
  <p>My EMBL file is only 1.19 GB compared to the GFF3 3.34 GB, so I probably lost a lot of annotation that is not supported by EMBL.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-03-10T00:40:53.310Z
Comments.Text: Please always mention your operating system and hardware. I am guessing that you are running this on a mac laptop, but it would be much easier if I didn't need to guess.
Comments.UserDisplayName: terdon
Tags: <file-formats><format-conversion><sequence-annotation><emboss>
Title: seqret error when trying to transform GFF3 and FASTA files into EMBL format
ThreadText: # seqret error when trying to transform GFF3 and FASTA files into EMBL format
  
  (Asked by: username_0 on 2018-03-09)
  
  I am using the tool [seqret](http://emboss.sourceforge.net/apps/release/6.6/emboss/apps/seqret.html) from emboss to transform an annotation file in GFF3 format and a FASTA file into an EMBL file because Wormbase does not supply an EMBL file with annotation and sequence together. I am running it on Mac OS X. I got the gff3 and fasta files from [Wormbase](ftp://ftp.wormbase.org/pub/wormbase/releases/current-production-release/species/c_elegans/PRJNA13758/). I installed emboss using conda from the bioconda channel. I use the following command:
  
  `seqret -sequence c_elegans.PRJNA13758.WS263.genomic.fa -feature -fformat gff -fopenfile c_elegans.PRJNA13758.WS263.annotations.gff3 -osformat embl -auto -outseq c_elegans.PRJNA13758.WS263.embl`
  
  It fails with the following error:
  
  ```
  /Users/me/anaconda/bin/seqret`: line 7: 54774 Killed: 9               _seqret "$@"
  
  ```
  
  `/Users/me/anaconda/bin/seqret` is a short file that indeed contains `_seqret "$@"` on line 7:
  
  ```
  #!/bin/sh
  BIN_DIR=$(dirname $(which seqret))
  export EMBOSS_ACDROOT=$BIN_DIR/../share/EMBOSS/acd/
  export EMBOSS_DATA=$BIN_DIR/../share/EMBOSS/data/
  export PLPLOT_LIB=$BIN_DIR/../share/EMBOSS/
  _seqret "$@"
  
  ```
  
  I tried to erase it but that did not help.
  
  I also tried [EMBLmyGFF3](https://github.com/NBISweden/EMBLmyGFF3) in a Python 2 environment:
  
  `EMBLmyGFF3 c_elegans.PRJNA13758.WS263.annotations.gff3 c_elegans.PRJNA13758.WS263.genomic.fa --topology linear --molecule_type 'genomic DNA' --transl_table 1 --species 'Caenorhabditis elegans' --locus_tag CELE --project_id PRJNA13758 -o c_elegans.PRJNA13758.WS263.annotations.embl`
  
  but I just had this message on display for hours:
  
  ```
  #############################################################################
  # NBIS 2016 - Sweden                                                        #
  # Authors: Martin Norling, Niclas Jareborg, Jacques Dainat                  #
  # Please visit https://github.com/NBISweden/EMBLmyGFF3 for more information #
  #############################################################################
  
  ```
  
  Update 11th March:
  
  I started EMBLmyGFF3 yesterday at 5 pm and this morning it started outputting messages such as:
  
  ```
  08:22:08 ERROR feature: >>inverted_repeat<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 ERROR feature: >>assembly_component<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 WARNING feature: Unknown qualifier 'genbank' - skipped
  08:22:08 ERROR feature: >>tandem_duplication<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 WARNING feature: Unknown qualifier 'public_name' - skipped
  08:22:08 WARNING feature: Unknown qualifier 'polymorphism' - skipped
  08:22:08 WARNING feature: Unknown qualifier 'other_name' - skipped
  08:22:08 WARNING feature: Unknown qualifier 'consequence' - skipped
  08:22:08 WARNING feature: Unknown qualifier 'variation' - skipped
  08:22:08 ERROR feature: >>biological_region<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 WARNING feature: Unknown qualifier 'balancer_type' - skipped
  08:22:08 WARNING feature: Unknown qualifier 'balancer' - skipped
  08:22:08 ERROR feature: >>protein_match<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 ERROR feature: >>transcript_region<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:22:08 ERROR feature: >>transcribed_fragment<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  
  ```
  
  ...
  
  ```
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:45:36 ERROR feature: >>piRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:46:00 ERROR feature: >>pseudogenic_tRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:47:59 ERROR feature: >>antisense_RNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:47:59 ERROR feature: >>lincRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:48:03 ERROR feature: >>pre_miRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:48:03 ERROR feature: >>miRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  08:50:37 ERROR feature: >>snRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  09:04:00 ERROR feature: >>miRNA_primary_transcript<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  
  ```
  
  For some of these features, it seems OK to not report them. But for others, such as piRNA, I need to find a solution. *C. elegans* has got 1000s of piRNAs, it would be nice to have them in the EMBL file. I need to find a list of valid EMBL feature types.
  
  Update + ~ 24h computation:
  
  ...
  
  ```
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  12:21:19 ERROR feature: >>scRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
  Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
  Conversion done
  
  ```
  
  My EMBL file is only 1.19 GB compared to the GFF3 3.34 GB, so I probably lost a lot of annotation that is not supported by EMBL.
  
  **Tags:** file-formats, format-conversion, sequence-annotation, emboss
  
  > **Comments:**
  > **username_1**: Please always mention your operating system and hardware. I am guessing that you are running this on a mac laptop, but it would be much easier if I didn't need to guess.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-03-10. Score: 2 | Accepted answer
  
  That error is telling you that the script you were running was killed by signal `9`, also known as `SIGKILL`, while executing the command `_seqret "$@"`. This is the way the operating system will kill processes that are asking for more memory than it can provide.
  
  Note that removing `_secret "$@"` is not the solution since that's what's actually doing the job. `$@` is a bash array that contains all the arguments passed to a bash script. The `/Users/me/anaconda/bin/seqret` file is just a simple script that sets up a few variables and then launches `_seqret`, passing it the arguments you gave to `/Users/me/anaconda/bin/seqret` as `$@`. So deleting that lines just makes the script useless.
  
  Based on your question, I am guessing you're running macOS on a laptop, so it isn't unreasonable that something that needs to read an entire genome and what looks like a full set of annotations will run out of memory on a laptop. Of course, this is a pretty small genome, but the annotations are still 396MB compressed, so depending on your machine's specs and what kind of internal processing the command is doing, it isn't too surprising that it fails.
  
  So, either run this on a more powerful machine, or try splitting the annotations file into smaller files and running it sequentially on each of those.
  
  
  ---
  
  ### Answer by username_2 on 2018-03-10. Score: 3
  
  I'm one of the EMBLmyGFF3 developers. If the compressed annotation is 396MB as mentioned by @username_1 it is not a small annotation... In the worse case we have tested we never waited more than few minutes. We haven't done any particular speed test. It would be valuable if you could tell us how long it has taken in your case (If you didn't kill the process before the end...).
  
  At least we could try to implement some nice features, like to see that the tool is working , or/and see which percentage of the file has been already processed.
  
  If it is really slow we could think to try to improve its speed.
  
  > **Comments:**
  > **username_0**: Does EMBLmyGFF3 accept .gz compressed files? Is it better? I have started the transformation at 5 pm today. The size of the uncompressed annotation file is 3.1 GB. The genome file is about 100 MB uncompressed. I'll let you know if/when it finishes. It would definitely be useful to have a progress bar and a sign that the tool is working properly. Thanks a lot!  
  > **username_0**: Still going and using 50 GB of memory (on a 16 GB physical RAM machine) but less than 100% of a CPU. So, it's using the hard disk to store some stuff.  
  > **username_0**: The process has taken about 24 hours in total on 16 GB physical RAM and 8 cores.  
  > **username_2**: Ok it is obviously really slow... Using .gz compressed file will not change anything here. We will investigate how to improve the speedness.  
  > **username_0**: I get a lot of errors of the type >>antisense_RNA<< is not a valid EMBL feature type. I found http://www.insdc.org/files/feature_table.html#2.3 but I could not locate a table with all the valid EMBL feature types. Do you know where I could find this? I get the error with miRNA as well, would ncRNA be the correct feature type in the EMBL format?  
  > **username_2**: You will find the complete list of features and qualifiers accepted by EBI here: https://www.ebi.ac.uk/ena/WebFeat/  
  
  ---
  
  