Answers.Body: <p>I'm afraid this is a clear bug in the tool you are using. The regular expression requires that all fields in the GTF file contain no whitespace. The <code>\S+</code> means &quot;one or more non-whitespace characters&quot; but your &quot;source&quot; field, the second field in the GTF file, contains spaces: <code>Desoria tigrina genome assembly, contig: utg000040l, whole genome shotgun sequence</code>.</p>
  <p>The thing is, that is a perfectly valid field for a GTF file! GTF fields can contain spaces, they just can't contain tabs. I tried to find a repository for the software to suggest a fix, but found only <a href="https://github.com/gatech-genemark/GeneMark-ES" rel="nofollow noreferrer">an empty one</a>. In any case, try changing the regex you show to this:</p>
  <pre class="lang-perl prettyprint-override"><code>#if ( /^(\S+)\t\S+\t(\S+)\t(\d+)\t(\d+)\t.\t([\+-])\t([012.])\tgene_id \&quot;(\S+)\&quot;/ )
  if ( /^([^\t]+)\t[^\t]+\t([^\t]+)\t(\d+)\t(\d+)\t.\t([\+-])\t([012.])\tgene_id \&quot;(\S+)\&quot;/ )
      {
      ... (some stuff)
      }
  </code></pre>
  <p>I can't guarantee that this will work as expected, please test! The details will depend on what they do with the captured groups (the <code>some stuff</code>), but at least that regex will match and I <em>think</em> I have kept the numbering of the capture groups.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-29T03:32:13.790Z
Answers.Text: I agree with this 100%. I read the regex and didn't spot the reply. There are quite a few problems, the use of a bare "." just means 'any character' rather '\.' which in this example would fit (albeit it might be intentional). The final section '; transcript_id "1_t";' isn't captured. However, the key problem is `\S` which will trip over the space characters in "CAJQTU010000037.1 Desoria tigrina genome assembly, contig: utg000040l, whole genome shotgun sequence". The use of '[^\t]', i.e. not a tab character will work. Personally I would split on the tab character.
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-29T03:39:50.190Z
Answers.Text: Just to clarify this "\t.\t" means any single character between two tabs. However, this "[012.]" might mean a `\.` i.e. just a full-stop, I can't remember. If it doesn't `[.]` is exactly the same as `[012.]` or even `[a-z|0-9]` and beyond. Anyway this is the accepted answer.
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-11T12:50:41.457Z
Answers.LastActivityDate: 2021-09-11T12:59:15.520Z
Answers.LastEditDate: 2021-09-11T12:59:15.520Z
Answers.UserDisplayName: terdon
Answers.Body: <p>You might use <code>agat_sp_extract_sequences.pl</code> from <a href="https://github.com/NBISweden/AGAT" rel="nofollow noreferrer">AGAT</a> instead.</p>
  <p>To extract and translate the coding regions:<br />
  <code>agat_sp_extract_sequences.pl -g infile.gxf -f infile.fasta -p</code></p>
  <p>see here for all possibilities with that script: <a href="https://agat.readthedocs.io/en/latest/tools/agat_sp_extract_sequences.html" rel="nofollow noreferrer">https://agat.readthedocs.io/en/latest/tools/agat_sp_extract_sequences.html</a></p>
  <p>Otherwise with the modification suggested by @terdon, your script should work fine.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-17T08:01:29.300Z
Answers.LastActivityDate: 2021-09-17T08:01:29.300Z
Answers.UserDisplayName: juke34
Body: <p>I am predicting genes using <code>GeneMark-ES</code>, I managed to generate a <code>gtf</code> file, but when I tried to extract the nt/protein sequence using <code>get_sequence_from_GTF.pl</code>, I am running into an error in the format of the gtf file, which I find puzzing as it was the gtf file was generated using <code>gmes_petap.pl</code> (an script in the <code>GeneMark-ES</code> package).</p>
  <p>Here is the command I used and the error message which also prints the first line of the genemark.gtf file:</p>
  <pre><code>$ get_sequence_from_GTF.pl genemark.gtf DT_CAJQTU010000037_1.fasta
  
  error in file format GTF: CAJQTU010000037.1 Desoria tigrina genome assembly, contig: utg000040l, whole genome shotgun sequence     GeneMark.hmm3   gene    2099    4039    .       +       .       gene_id &quot;1_g&quot;; transcript_id &quot;1_t&quot;;
  </code></pre>
  <p>I think I have found the line in <code>get_sequence_from_GTF.pl</code> that is causing the error, but not being a perl monk, I am having troubles to read figure out what is wrong with my <code>gtf</code> file and more importantly figure out how to fix it.</p>
  <pre><code>...
  if ( /^(\S+)\t\S+\t(\S+)\t(\d+)\t(\d+)\t.\t([\+-])\t([012.])\tgene_id \&quot;(\S+)\&quot;/ )
      {
      ... (some stuff)
      }
  else
  {
    if ( /^\s+<span class="math-container">$/ ) { next };
    if ( /^#/ ) { next };
    die( "error in file format GTF: $</span>_\n&quot; );
  }
  ... 
  </code></pre>
  
Tags: <gtf>
Title: GTF file format causing GeneMark-ES error
ThreadText: # GTF file format causing GeneMark-ES error
  
  (Asked by: username_0 on 2021-09-03)
  
  I am predicting genes using `GeneMark-ES`, I managed to generate a `gtf` file, but when I tried to extract the nt/protein sequence using `get_sequence_from_GTF.pl`, I am running into an error in the format of the gtf file, which I find puzzing as it was the gtf file was generated using `gmes_petap.pl` (an script in the `GeneMark-ES` package).
  
  Here is the command I used and the error message which also prints the first line of the genemark.gtf file:
  
  ```
  $ get_sequence_from_GTF.pl genemark.gtf DT_CAJQTU010000037_1.fasta
  
  error in file format GTF: CAJQTU010000037.1 Desoria tigrina genome assembly, contig: utg000040l, whole genome shotgun sequence     GeneMark.hmm3   gene    2099    4039    .       +       .       gene_id "1_g"; transcript_id "1_t";
  
  ```
  
  I think I have found the line in `get_sequence_from_GTF.pl` that is causing the error, but not being a perl monk, I am having troubles to read figure out what is wrong with my `gtf` file and more importantly figure out how to fix it.
  
  ```
  ...
  if ( /^(\S+)\t\S+\t(\S+)\t(\d+)\t(\d+)\t.\t([\+-])\t([012.])\tgene_id \"(\S+)\"/ )
      {
      ... (some stuff)
      }
  else
  {
    if ( /^\s+$/ ) { next };
    if ( /^#/ ) { next };
    die( "error in file format GTF: $_\n" );
  }
  ... 
  
  ```
  
  **Tags:** gtf
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2021-09-11. Score: 2
  
  I'm afraid this is a clear bug in the tool you are using. The regular expression requires that all fields in the GTF file contain no whitespace. The `\S+` means "one or more non-whitespace characters" but your "source" field, the second field in the GTF file, contains spaces: `Desoria tigrina genome assembly, contig: utg000040l, whole genome shotgun sequence`.
  
  The thing is, that is a perfectly valid field for a GTF file! GTF fields can contain spaces, they just can't contain tabs. I tried to find a repository for the software to suggest a fix, but found only [an empty one](https://github.com/gatech-genemark/GeneMark-ES). In any case, try changing the regex you show to this:
  
  ```perl
  #if ( /^(\S+)\t\S+\t(\S+)\t(\d+)\t(\d+)\t.\t([\+-])\t([012.])\tgene_id \"(\S+)\"/ )
  if ( /^([^\t]+)\t[^\t]+\t([^\t]+)\t(\d+)\t(\d+)\t.\t([\+-])\t([012.])\tgene_id \"(\S+)\"/ )
      {
      ... (some stuff)
      }
  
  ```
  
  I can't guarantee that this will work as expected, please test! The details will depend on what they do with the captured groups (the `some stuff`), but at least that regex will match and I *think* I have kept the numbering of the capture groups.
  
  > **Comments:**
  > **username_2**: I agree with this 100%. I read the regex and didn't spot the reply. There are quite a few problems, the use of a bare "." just means 'any character' rather '\.' which in this example would fit (albeit it might be intentional). The final section '; transcript_id "1_t";' isn't captured. However, the key problem is `\S` which will trip over the space characters in "CAJQTU010000037.1 Desoria tigrina genome assembly, contig: utg000040l, whole genome shotgun sequence". The use of '[^\t]', i.e. not a tab character will work. Personally I would split on the tab character.  
  > **username_2**: Just to clarify this "\t.\t" means any single character between two tabs. However, this "[012.]" might mean a `\.` i.e. just a full-stop, I can't remember. If it doesn't `[.]` is exactly the same as `[012.]` or even `[a-z|0-9]` and beyond. Anyway this is the accepted answer.  
  
  ---
  
  ### Answer by username_3 on 2021-09-17. Score: 1
  
  You might use `agat_sp_extract_sequences.pl` from [AGAT](https://github.com/NBISweden/AGAT) instead.
  
  To extract and translate the coding regions:  
  `agat_sp_extract_sequences.pl -g infile.gxf -f infile.fasta -p`
  
  see here for all possibilities with that script: <https://agat.readthedocs.io/en/latest/tools/agat_sp_extract_sequences.html>
  
  Otherwise with the modification suggested by @username_1, your script should work fine.
  
  
  ---
  
  