Answers.Body: <p>After trying myself in many ways, I have found that the method of creating a checkmark column when a desired column is recorded is more efficient and easier to understand.</p>
  <p>Here is my bash script to solve my problem in a verbose way (I guess has others way shorter).</p>
  <pre class="lang-bash prettyprint-override"><code>#!/usr/bin/bash
  
  change_index(){
      local in=&quot;<span class="math-container">$(&lt; /dev/stdin)" # Variable assignment from the stdin
      local col_num=$</span>(head -n 1 &lt;(echo &quot;$in&quot;) |awk -F&quot;\t&quot; '{print -1+NF/2}')  # Number of fields
  
      
      for k in <span class="math-container">$(seq 0 $</span>((<span class="math-container">$col_num-1))); do # Loop through number of field
          in=$</span>(awk -v ia=&quot;<span class="math-container">$k" -F"\t"  '{sub("/[^"ia"]","/"ia"\t-\t/"((ia+1)),$(2*ia+1)) ;print $0}' &lt;(echo "$</span>in&quot;)) # Supplement missing fields
      done
      
      in=<span class="math-container">$(awk -v ia="$</span>col_num&quot; -F&quot;\t&quot;  '{sub(&quot;/[^&quot;ia&quot;]&quot;,&quot;/&quot;ia&quot;\t-&quot;,<span class="math-container">$(2*ia+1)) ;print $</span>0}' &lt;(echo &quot;<span class="math-container">$in")) # Supplement missing the last fields
      in=$</span>(sed 's/\/[[:digit:]]\+//g' &lt;(echo &quot;<span class="math-container">$in")) # Remove markers
      echo "$</span>in&quot;
  }
  
  for i in {U47804 U47803 U47802 U47801 U47800 U47799 U47798 X92938}; do \
      efetch -db &quot;nucleotide&quot; -id &quot;$i&quot; -format full; done |    
      xtract -pattern Seq-entry \
          -lbl &quot;/0&quot; -element Org-ref_taxname \
          -block Prot-ref_name -lbl &quot;/1&quot; -element Prot-ref_name_E \
          -block Date-std -position first -lbl &quot;/2&quot; -element Date-std_year |
      change_index| column -t
  </code></pre>
  <h2>Explanation</h2>
  <p>The main idea is that if a field is recorded there will be a marker field lay before the recorded field. This action is done by <code>-lbl</code> tag. And we have the following draft table:</p>
  <pre><code>/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /2  1995       
  </code></pre>
  <p>After having a draft table, and cleaning the table by <code>change_index()</code> function I wrote.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-17T11:59:41.730Z
Answers.LastActivityDate: 2023-02-18T08:53:58.943Z
Answers.LastEditDate: 2023-02-18T08:53:58.943Z
Answers.UserDisplayName: Rossy Clair
Body: <p>I am using the <a href="https://dataguide.nlm.nih.gov/edirect/xtract.html" rel="nofollow noreferrer">xtract tool</a> of the Entrez Direct (<a href="https://www.ncbi.nlm.nih.gov/books/NBK179288/" rel="nofollow noreferrer">EDirect</a>) package of NBCI. I have a list of Accession Numbers:</p>
  <pre><code>U47804, U47803, U47802, U47801, U47800, U47799, U47798, X92938
  </code></pre>
  <p>and try to get information about:</p>
  <ol>
  <li>Name of organism (Org-ref_taxname tag)</li>
  <li>Product of gene (Prot-ref_name_E tag)</li>
  <li>Public year (Date-std_year tag)</li>
  </ol>
  <p>I executed the following command:</p>
  <pre class="lang-bash prettyprint-override"><code>for i in {U47804 U47803 U47802 U47801 U47800 U47799 U47798 X92938}
  do
    efetch -db &quot;nucleotide&quot; -id &quot;$i&quot; -format full; done |
      xtract -pattern Seq-entry -element Org-ref_taxname \
        -block Prot-ref_name -def &quot;-&quot; -element Prot-ref_name_E \
        -block Date-std -position first -element Date-std_year
  </code></pre>
  <p>The result I got:</p>
  <pre><code>Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  1995
  </code></pre>
  <p>Because the <strong>X92938</strong> doesn't have <em>&lt;Prot-ref_name&gt;</em>, so I was expecting to replace the missing fields with <strong>&quot;-&quot;</strong> by the <code>-def</code> tag of the extract tool. It means my desired result is as follows:</p>
  <pre><code>Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1    -     1995
  </code></pre>
  <p>Can someone have experience with the NBCI tool may help me? Thanks a lot.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-16T23:40:45.760Z
Comments.Text: I saw this on [stackoverflow](https://stackoverflow.com/questions/75469564/default-placeholder-for-missing-fields-doesnt-work-with-block-in-xtract-tool-b) but I couldn't figure out the problem. One of the great things about https://bioinformatics.stackexchange.com/ is that you can include heaps more detail here; if you could please provide more context it might help e.g. Why do you want this data? Have you looked at other sources e.g. [the Rentrez package](https://cran.r-project.org/web/packages/rentrez/vignettes/rentrez_tutorial.html)? Could you post-format the data using e.g. `awk`? Etc
Comments.UserDisplayName: jared_mamrot
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-19T02:37:33.397Z
Comments.Text: Thanks for your helpful suggestion. According to your suggestion, I have used [Rentrez](https://cran.r-project.org/web/packages/rentrez/vignettes/rentrez_tutorial.html#web_history) and realized it is much easier to use and more powerful than the Entrez bash. So in some special cases like creating bioinformatics pipelines, using bash Entrez is a compulsory thing.
Comments.UserDisplayName: Rossy Clair
Tags: <bash><edirect>
Title: Default placeholder for missing fields doesn't work with block in xtract tool
ThreadText: # Default placeholder for missing fields doesn't work with block in xtract tool
  
  (Asked by: username_0 on 2023-02-16)
  
  I am using the [xtract tool](https://dataguide.nlm.nih.gov/edirect/xtract.html) of the Entrez Direct ([EDirect](https://www.ncbi.nlm.nih.gov/books/NBK179288/)) package of NBCI. I have a list of Accession Numbers:
  
  ```
  U47804, U47803, U47802, U47801, U47800, U47799, U47798, X92938
  
  ```
  
  and try to get information about:
  
  1. Name of organism (Org-ref\_taxname tag)
  2. Product of gene (Prot-ref\_name\_E tag)
  3. Public year (Date-std\_year tag)
  
  I executed the following command:
  
  ```bash
  for i in {U47804 U47803 U47802 U47801 U47800 U47799 U47798 X92938}
  do
    efetch -db "nucleotide" -id "$i" -format full; done |
      xtract -pattern Seq-entry -element Org-ref_taxname \
        -block Prot-ref_name -def "-" -element Prot-ref_name_E \
        -block Date-std -position first -element Date-std_year
  
  ```
  
  The result I got:
  
  ```
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  1995
  
  ```
  
  Because the **X92938** doesn't have *<Prot-ref\_name>*, so I was expecting to replace the missing fields with **"-"** by the `-def` tag of the extract tool. It means my desired result is as follows:
  
  ```
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1  gp120   1996
  Human immunodeficiency virus 1    -     1995
  
  ```
  
  Can someone have experience with the NBCI tool may help me? Thanks a lot.
  
  **Tags:** bash, edirect
  
  > **Comments:**
  > **username_1**: I saw this on [stackoverflow](https://stackoverflow.com/questions/75469564/default-placeholder-for-missing-fields-doesnt-work-with-block-in-xtract-tool-b) but I couldn't figure out the problem. One of the great things about https://bioinformatics.stackexchange.com/ is that you can include heaps more detail here; if you could please provide more context it might help e.g. Why do you want this data? Have you looked at other sources e.g. [the Rentrez package](https://cran.r-project.org/web/packages/rentrez/vignettes/rentrez_tutorial.html)? Could you post-format the data using e.g. `awk`? Etc  
  > **username_0**: Thanks for your helpful suggestion. According to your suggestion, I have used [Rentrez](https://cran.r-project.org/web/packages/rentrez/vignettes/rentrez_tutorial.html#web_history) and realized it is much easier to use and more powerful than the Entrez bash. So in some special cases like creating bioinformatics pipelines, using bash Entrez is a compulsory thing.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2023-02-17. Score: 2 | Accepted answer
  
  After trying myself in many ways, I have found that the method of creating a checkmark column when a desired column is recorded is more efficient and easier to understand.
  
  Here is my bash script to solve my problem in a verbose way (I guess has others way shorter).
  
  ```bash
  #!/usr/bin/bash
  
  change_index(){
      local in="$(< /dev/stdin)" # Variable assignment from the stdin
      local col_num=$(head -n 1 <(echo "$in") |awk -F"\t" '{print -1+NF/2}')  # Number of fields
  
      
      for k in $(seq 0 $(($col_num-1))); do # Loop through number of field
          in=$(awk -v ia="$k" -F"\t"  '{sub("/[^"ia"]","/"ia"\t-\t/"((ia+1)),$(2*ia+1)) ;print $0}' <(echo "$in")) # Supplement missing fields
      done
      
      in=$(awk -v ia="$col_num" -F"\t"  '{sub("/[^"ia"]","/"ia"\t-",$(2*ia+1)) ;print $0}' <(echo "$in")) # Supplement missing the last fields
      in=$(sed 's/\/[[:digit:]]\+//g' <(echo "$in")) # Remove markers
      echo "$in"
  }
  
  for i in {U47804 U47803 U47802 U47801 U47800 U47799 U47798 X92938}; do \
      efetch -db "nucleotide" -id "$i" -format full; done |    
      xtract -pattern Seq-entry \
          -lbl "/0" -element Org-ref_taxname \
          -block Prot-ref_name -lbl "/1" -element Prot-ref_name_E \
          -block Date-std -position first -lbl "/2" -element Date-std_year |
      change_index| column -t
  
  ```
  
  ## Explanation
  
  The main idea is that if a field is recorded there will be a marker field lay before the recorded field. This action is done by `-lbl` tag. And we have the following draft table:
  
  ```
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996
  /0  Human  immunodeficiency  virus  1  /2  1995       
  
  ```
  
  After having a draft table, and cleaning the table by `change_index()` function I wrote.
  
  
  ---
  
  