Answers.Body: <p>Regarding your <code>grep</code>, why not just grep for what you want, <em>i.e.</em> <code>^ENSG.*</code>? (These last few lines that htseq-count outputs - which is the software that produces your files - are really important though. You should absolutely look into them to see if something strange is going on with one or more samples.)</p>
  <p>Your error comes from the escaped quotes <code>\&quot;</code> you use here:</p>
  <pre><code>out.file2 &lt;- merge(out.file, file3, by=c(\&quot;gene_id\&quot;))
  </code></pre>
  <p>Before, you used:</p>
  <pre><code>out.file = merge (file1, file2, by=c(&quot;gene_id&quot;))
  </code></pre>
  <p>which is the correct way to quote here. On a sidenote, you don't need <code>c()</code> if it's only a vector with one element. You can just write <code>by = &quot;gene_id&quot;</code>.</p>
  <p>Here's a different way you could read the files:</p>
  <pre><code>files &lt;- list.files(path=&quot;.&quot;, pattern=&quot;GSM&quot;)
  count_files &lt;- lapply(files, read.table, col.names = c(&quot;GeneID&quot;, &quot;Count&quot;))
  count_matrix &lt;- do.call(cbind, lapply(count_files, &quot;[[&quot;, &quot;Count&quot;))
  dimnames(count_matrix) &lt;- list(count_files[[1]][&quot;GeneID&quot;], files)
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-26T18:32:35.930Z
Answers.LastActivityDate: 2021-07-23T21:52:56.403Z
Answers.LastEditDate: 2021-07-23T21:52:56.403Z
Answers.UserDisplayName: Bastian Schiffthaler
Body: <p>I have these files that I want to make table of (<a href="https://github.com/learnseq/RNAseqfiles01.git" rel="nofollow noreferrer">https://github.com/learnseq/RNAseqfiles01.git</a>), the table that I want (since all the files have the same ID column),, I want merge the ID column with the count column from each file:</p>
  <pre><code>ID    File 1 count column     File 2 count column     File 3 count column   File 4 count column
  
  </code></pre>
  <p>I cleaned up the files from extra lines using:</p>
  <pre><code>grep -vE '^{' GSM2580324_counts.txt | grep -vE '^\\' | grep -v '^ *$' | sed '/^{/d'  | tr '\\' ' ' | sed '/}/d' | grep -vE '^__' &gt; GSM2580324_counts04.txt
  </code></pre>
  <p>But when I try to make a table in R using this:</p>
  <pre><code>%R files = list.files(path=&quot;.&quot;, pattern=&quot;GSM&quot;)
  %R file1 = read.table(&quot;GSM2580319_counts01.txt&quot; , col.names=c(&quot;gene_id&quot;,&quot;GSM2580319_counts01.txt&quot; ))
  %R file2 = read.table(&quot;GSM2580320_counts02.txt&quot; , col.names=c(&quot;gene_id&quot;,&quot;GSM2580320_counts02.txt&quot; ))
  %R file3 = read.table(&quot;GSM2580323_counts03.txt&quot;, col.names=c(&quot;gene_id&quot;,&quot;GSM2580323_counts03.txt&quot; ))
  %R file4 = read.table(&quot;GSM2580324_counts04.txt&quot;, col.names=c(&quot;gene_id&quot;,&quot;GSM2580324_counts04.txt&quot;))
  %R out.file = merge (file1, file2, by=c(&quot;gene_id&quot;))
  %R out.file2 &lt;- merge(out.file, file3, by=c(\&quot;gene_id\&quot;))
  %R out.file3 &lt;- merge(out.file2, file4, by=c(\&quot;gene_id\&quot;))
  %R outtable01 &lt;- write.table(out.file, file = &quot;out.file3&quot;,sep=&quot;\t&quot;, row.names = FALSE)
  %R outtable01                        
  
  </code></pre>
  <p>I get this error:</p>
  <pre><code>RParsingError                             Traceback (most recent call last)
  &lt;ipython-input-179-2c471cf65113&gt; in &lt;module&gt;
        7 get_ipython().run_line_magic('R', 'file4 = read.table(&quot;GSM2580324_counts04.txt&quot;, col.names=c(&quot;gene_id&quot;,&quot;GSM2580324_counts04.txt&quot;))')
        8 get_ipython().run_line_magic('R', 'out.file = merge (file1, file2, by=c(&quot;gene_id&quot;))')
  ----&gt; 9 get_ipython().run_line_magic('R', 'out.file2 &lt;- merge(out.file, file3, by=c(\\&quot;gene_id\\&quot;))')
       10 get_ipython().run_line_magic('R', 'out.file3 &lt;- merge(out.file2, file4, by=c(\\&quot;gene_id\\&quot;))')
       11 get_ipython().run_line_magic('R', 'outtable01 &lt;- write.table(out.file, file = &quot;out.file3&quot;,sep=&quot;\\t&quot;, row.names = FALSE)')
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
     2315                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
     2316             with self.builtin_trap:
  -&gt; 2317                 result = fn(*args, **kwargs)
     2318             return result
     2319 
  
  &lt;decorator-gen-160&gt; in R(self, line, cell, local_ns)
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/IPython/core/magic.py in &lt;lambda&gt;(f, *a, **k)
      185     # but it's overkill for just that one bit of state.
      186     def magic_deco(arg):
  --&gt; 187         call = lambda f, *a, **k: f(*a, **k)
      188 
      189         if callable(arg):
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/ipython/rmagic.py in R(self, line, cell, local_ns)
      729             if line_mode:
      730                 for line in code.split(';'):
  --&gt; 731                     text_result, result, visible = self.eval(line)
      732                     text_output += text_result
      733                 if text_result:
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/ipython/rmagic.py in eval(self, code)
      262             try:
      263                 # Need the newline in case the last line in code is a comment.
  --&gt; 264                 value, visible = ro.r(&quot;withVisible({%s\n})&quot; % code)
      265             except (ri.embedded.RRuntimeError, ValueError) as exception:
      266                 # Otherwise next return seems to have copy of error.
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/robjects/__init__.py in __call__(self, string)
      385 
      386     def __call__(self, string):
  --&gt; 387         p = rinterface.parse(string)
      388         res = self.eval(p)
      389         return conversion.rpy2py(res)
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/rinterface_lib/conversion.py in _(*args, **kwargs)
       38 def _cdata_res_to_rinterface(function):
       39     def _(*args, **kwargs):
  ---&gt; 40         cdata = function(*args, **kwargs)
       41         # TODO: test cdata is of the expected CType
       42         return _cdata_to_rinterface(cdata)
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/rinterface.py in parse(text, num)
       44         raise TypeError('text must be a string.')
       45     robj = StrSexpVector([text])
  ---&gt; 46     return _rinterface._parse(robj.__sexp__._cdata, num)
       47 
       48 
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/rinterface_lib/_rinterface_capi.py in _parse(cdata, num)
      556         if status[0] != openrlib.rlib.PARSE_OK:
      557             raise RParsingError('Parsing status not OK',
  --&gt; 558                                 PARSING_STATUS(status[0]))
      559     return res
  
  RParsingError: Parsing status not OK
  </code></pre>
  <p>What should I do?</p>
  
Tags: <r><rna-seq><bash>
Title: Table error when combining counts columns
ThreadText: # Table error when combining counts columns
  
  (Asked by: username_0 on 2020-10-26)
  
  I have these files that I want to make table of (<https://github.com/learnseq/RNAseqfiles01.git>), the table that I want (since all the files have the same ID column),, I want merge the ID column with the count column from each file:
  
  ```
  ID    File 1 count column     File 2 count column     File 3 count column   File 4 count column
  
  ```
  
  I cleaned up the files from extra lines using:
  
  ```
  grep -vE '^{' GSM2580324_counts.txt | grep -vE '^\\' | grep -v '^ *$' | sed '/^{/d'  | tr '\\' ' ' | sed '/}/d' | grep -vE '^__' > GSM2580324_counts04.txt
  
  ```
  
  But when I try to make a table in R using this:
  
  ```
  %R files = list.files(path=".", pattern="GSM")
  %R file1 = read.table("GSM2580319_counts01.txt" , col.names=c("gene_id","GSM2580319_counts01.txt" ))
  %R file2 = read.table("GSM2580320_counts02.txt" , col.names=c("gene_id","GSM2580320_counts02.txt" ))
  %R file3 = read.table("GSM2580323_counts03.txt", col.names=c("gene_id","GSM2580323_counts03.txt" ))
  %R file4 = read.table("GSM2580324_counts04.txt", col.names=c("gene_id","GSM2580324_counts04.txt"))
  %R out.file = merge (file1, file2, by=c("gene_id"))
  %R out.file2 <- merge(out.file, file3, by=c(\"gene_id\"))
  %R out.file3 <- merge(out.file2, file4, by=c(\"gene_id\"))
  %R outtable01 <- write.table(out.file, file = "out.file3",sep="\t", row.names = FALSE)
  %R outtable01                        
  
  ```
  
  I get this error:
  
  ```
  RParsingError                             Traceback (most recent call last)
  <ipython-input-179-2c471cf65113> in <module>
        7 get_ipython().run_line_magic('R', 'file4 = read.table("GSM2580324_counts04.txt", col.names=c("gene_id","GSM2580324_counts04.txt"))')
        8 get_ipython().run_line_magic('R', 'out.file = merge (file1, file2, by=c("gene_id"))')
  ----> 9 get_ipython().run_line_magic('R', 'out.file2 <- merge(out.file, file3, by=c(\\"gene_id\\"))')
       10 get_ipython().run_line_magic('R', 'out.file3 <- merge(out.file2, file4, by=c(\\"gene_id\\"))')
       11 get_ipython().run_line_magic('R', 'outtable01 <- write.table(out.file, file = "out.file3",sep="\\t", row.names = FALSE)')
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
     2315                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
     2316             with self.builtin_trap:
  -> 2317                 result = fn(*args, **kwargs)
     2318             return result
     2319 
  
  <decorator-gen-160> in R(self, line, cell, local_ns)
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
      185     # but it's overkill for just that one bit of state.
      186     def magic_deco(arg):
  --> 187         call = lambda f, *a, **k: f(*a, **k)
      188 
      189         if callable(arg):
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/ipython/rmagic.py in R(self, line, cell, local_ns)
      729             if line_mode:
      730                 for line in code.split(';'):
  --> 731                     text_result, result, visible = self.eval(line)
      732                     text_output += text_result
      733                 if text_result:
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/ipython/rmagic.py in eval(self, code)
      262             try:
      263                 # Need the newline in case the last line in code is a comment.
  --> 264                 value, visible = ro.r("withVisible({%s\n})" % code)
      265             except (ri.embedded.RRuntimeError, ValueError) as exception:
      266                 # Otherwise next return seems to have copy of error.
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/robjects/__init__.py in __call__(self, string)
      385 
      386     def __call__(self, string):
  --> 387         p = rinterface.parse(string)
      388         res = self.eval(p)
      389         return conversion.rpy2py(res)
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/rinterface_lib/conversion.py in _(*args, **kwargs)
       38 def _cdata_res_to_rinterface(function):
       39     def _(*args, **kwargs):
  ---> 40         cdata = function(*args, **kwargs)
       41         # TODO: test cdata is of the expected CType
       42         return _cdata_to_rinterface(cdata)
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/rinterface.py in parse(text, num)
       44         raise TypeError('text must be a string.')
       45     robj = StrSexpVector([text])
  ---> 46     return _rinterface._parse(robj.__sexp__._cdata, num)
       47 
       48 
  
  /opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/rinterface_lib/_rinterface_capi.py in _parse(cdata, num)
      556         if status[0] != openrlib.rlib.PARSE_OK:
      557             raise RParsingError('Parsing status not OK',
  --> 558                                 PARSING_STATUS(status[0]))
      559     return res
  
  RParsingError: Parsing status not OK
  
  ```
  
  What should I do?
  
  **Tags:** r, rna-seq, bash
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-10-26. Score: 1
  
  Regarding your `grep`, why not just grep for what you want, *i.e.* `^ENSG.*`? (These last few lines that htseq-count outputs - which is the software that produces your files - are really important though. You should absolutely look into them to see if something strange is going on with one or more samples.)
  
  Your error comes from the escaped quotes `\"` you use here:
  
  ```
  out.file2 <- merge(out.file, file3, by=c(\"gene_id\"))
  
  ```
  
  Before, you used:
  
  ```
  out.file = merge (file1, file2, by=c("gene_id"))
  
  ```
  
  which is the correct way to quote here. On a sidenote, you don't need `c()` if it's only a vector with one element. You can just write `by = "gene_id"`.
  
  Here's a different way you could read the files:
  
  ```
  files <- list.files(path=".", pattern="GSM")
  count_files <- lapply(files, read.table, col.names = c("GeneID", "Count"))
  count_matrix <- do.call(cbind, lapply(count_files, "[[", "Count"))
  dimnames(count_matrix) <- list(count_files[[1]]["GeneID"], files)
  
  ```
  
  
  ---
  
  