Answers.Body: <p>I believe a <code>pandas</code> approach would be fast enough (probably faster than any other Python approach without pandas).</p>
  
  <p>After converting your table to a <code>pandas data frame</code>, you can do something like <code>df.filter(regex=(".*Chr01"))</code> for the first chromosome then write the resulting data to file with the <code>to_csv()</code> method.</p>
  
  <pre><code>import pandas as pd
  
  data = pd.read_csv('input.txt', delimiter = "\t", index_col = False)
  
  for i in ["Chr01", "Chr02", "Chr03"]:
      regex_pattern = ".*" + i
      print(data.filter(regex=regex_pattern))
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-10T07:28:02.960Z
Answers.LastActivityDate: 2019-10-10T21:44:46.147Z
Answers.LastEditDate: 2019-10-10T21:44:46.147Z
Answers.UserDisplayName: haci
Body: <p>I have already asked this question in another forum but have not got an appropriate answer, so wondering if anyone here can help me?
  I have a big file with over 3 million columns (SNPs genotypes) and 800 rows (one row for each individual). </p>
  
  <p>The first row is the SNPs_ID name with the SNP position on each chromosome (see the example below please), in total there are 17 chromosomes. I want to extract SNPs for each chromosome and store them in a separate file.
  I have done it with "R", but I am looking for a faster solution and I am wondering if anyone can help me to figure out another solution with bash, perl or python codes?</p>
  
  <p>here is an example of my input file (SNPs on chromosome 1,2,3,4...,17)</p>
  
  <pre><code>"Ha412HOChr01:180159" "Ha412HOChr01:210724" "Ha412HOChr01:303270" "Ha412HOChr01:303280"....... "Ha412HOChr17:303402"
  0 1 0 0 ......0
  0 1 0 0 ......0
  0 1 0 0 ......0
  0 2 0 0 ......0
  0 1 1 1 ......1
  0 2 0 0 ......0
  </code></pre>
  
  <p>and this is an example of my desired output for example for chromosome 1:</p>
  
  <pre><code>out.chrom1
  "Ha412HOChr01:180159" "Ha412HOChr01:210724" "Ha412HOChr01:303270" "Ha412HOChr01:303280" 
  0 1 0 0 
  0 1 0 0
  0 1 0 0 
  0 2 0 0 
  0 1 1 1 
  0 2 0 0 
  0 0 0 0 
  0 2 0 0
  0 1 2 2 
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-10T04:32:32.587Z
Comments.Text: Hi, good question but perhaps post your R code?
Comments.UserDisplayName: M__
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-10T14:27:28.167Z
Comments.Text: Are the leading spaces part of your input file? Does the second row, for example really start with 4 spaces? If not, please [edit] and remove them to avoid confusion. Also, are those really the IDs? Do they all start with `Ha412HO`, then `Chr` and number(s), followed by `:` and the position? Or can the `Ha412HO` change?
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-10T15:26:47.920Z
Comments.Text: I just edited my question according to your comment. And yes, the IDs start with Ha412HOChr followed by chromosome, the : and SNP position
Comments.UserDisplayName: Anna1364
Tags: <python><perl>
Title: storing SNPs in the genotype file for each chromosome in a separate file
ThreadText: # storing SNPs in the genotype file for each chromosome in a separate file
  
  (Asked by: username_0 on 2019-10-10)
  
  I have already asked this question in another forum but have not got an appropriate answer, so wondering if anyone here can help me?
  I have a big file with over 3 million columns (SNPs genotypes) and 800 rows (one row for each individual).
  
  The first row is the SNPs\_ID name with the SNP position on each chromosome (see the example below please), in total there are 17 chromosomes. I want to extract SNPs for each chromosome and store them in a separate file.
  I have done it with "R", but I am looking for a faster solution and I am wondering if anyone can help me to figure out another solution with bash, perl or python codes?
  
  here is an example of my input file (SNPs on chromosome 1,2,3,4...,17)
  
  ```
  "Ha412HOChr01:180159" "Ha412HOChr01:210724" "Ha412HOChr01:303270" "Ha412HOChr01:303280"....... "Ha412HOChr17:303402"
  0 1 0 0 ......0
  0 1 0 0 ......0
  0 1 0 0 ......0
  0 2 0 0 ......0
  0 1 1 1 ......1
  0 2 0 0 ......0
  
  ```
  
  and this is an example of my desired output for example for chromosome 1:
  
  ```
  out.chrom1
  "Ha412HOChr01:180159" "Ha412HOChr01:210724" "Ha412HOChr01:303270" "Ha412HOChr01:303280" 
  0 1 0 0 
  0 1 0 0
  0 1 0 0 
  0 2 0 0 
  0 1 1 1 
  0 2 0 0 
  0 0 0 0 
  0 2 0 0
  0 1 2 2 
  
  ```
  
  **Tags:** python, perl
  
  > **Comments:**
  > **username_1**: Hi, good question but perhaps post your R code?  
  > **username_2**: Are the leading spaces part of your input file? Does the second row, for example really start with 4 spaces? If not, please [edit] and remove them to avoid confusion. Also, are those really the IDs? Do they all start with `Ha412HO`, then `Chr` and number(s), followed by `:` and the position? Or can the `Ha412HO` change?  
  > **username_0**: I just edited my question according to your comment. And yes, the IDs start with Ha412HOChr followed by chromosome, the : and SNP position  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-10-10. Score: 2
  
  I believe a `pandas` approach would be fast enough (probably faster than any other Python approach without pandas).
  
  After converting your table to a `pandas data frame`, you can do something like `df.filter(regex=(".*Chr01"))` for the first chromosome then write the resulting data to file with the `to_csv()` method.
  
  ```
  import pandas as pd
  
  data = pd.read_csv('input.txt', delimiter = "\t", index_col = False)
  
  for i in ["Chr01", "Chr02", "Chr03"]:
      regex_pattern = ".*" + i
      print(data.filter(regex=regex_pattern))
  
  ```
  
  
  ---
  
  