Answers.Body: <p>I can see what you're wanting to do, I can see bugs but the key bit of output is missing (below). It seems a bit weird to me the nested loop in context, but thats fine if you're happy.</p>
  <p>The critical step is this</p>
  <pre><code>df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)
  </code></pre>
  <p>What I'm concerned about is the <code>pd.concat</code>. Are you sure <code>pd.concat</code> is the right command here? I thought <code>m</code> is one data table, e.g. mutation and <code>g</code> is another e.g. expression. <code>pd.concat</code> isn't going to work here, are you sure this isn't a <code>merge</code> or <code>join</code> command?</p>
  <p>Anyway, the fitting is complaining the <code>.shape()</code> is wrong. Needs reshaping? Thats the error. Printing <code>df</code> at that point would be needed.</p>
  <blockquote>
  <p>ValueError: endog has evaluated to an array with multiple columns that has shape (10, 18)</p>
  </blockquote>
  <p>To summarise normally the error you have received is about re-shaping the data. I've not put forward that solution here because,</p>
  <ul>
  <li>I am not sure what the data represents</li>
  <li>If I was doing this I wouldn't use <code>pd.concat</code></li>
  </ul>
  <p>It seems a lot of columns for a 1-way ANOVA, but could be right, I've not done 1-way ANOVA across 10 columns.</p>
  <p><em>Bug 1</em></p>
  <pre><code>m = pd.read_csv(&quot;mut.csv&quot;, sep=&quot;,&quot;)
  m=pd.DataFrame(m)
  m.info()
  g = pd.read_csv(&quot;expr1.csv&quot;, sep=&quot;,&quot;)
  g=pd.DataFrame(g)
  </code></pre>
  <p>This should simply be:</p>
  <pre><code>m = pd.read_csv(&quot;mut.csv&quot;)
  g = pd.read_csv(&quot;expr1.csv&quot;)
  </code></pre>
  <p><em>Bug 2?</em></p>
  <pre><code>dfs = pd.DataFrame({})
  ...
  dfs = dfs.append(aov_table)
  </code></pre>
  <p>This will work given <code>aov_table</code> output is a pandas dataframe (I'm not at all sure it is). You need <code>aov_table.info()</code> to be sure. You can always place it in a dataframe however.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-27T18:35:08.180Z
Answers.LastActivityDate: 2023-09-27T23:01:10.543Z
Answers.LastEditDate: 2023-09-27T23:01:10.543Z
Answers.UserDisplayName: M__
Body: <p>I have mutation status (<code>0</code>,<code>1</code>) for ten genes in ten cancer 3D models</p>
  <p>I have also gene expression in these 3D models, like</p>
  <pre class="lang-r prettyprint-override"><code>expr1 &lt;- structure(list(GeneA_KO = c(0.0756383119639074, -0.162841965682723, -0.110703100204517, -0.457154616239399, -0.505815193885352, 0.366072775033448, 0.288650458935083, -0.0287646498640309, -0.690732824311905, 0.157366122620995, -1.29451546737684, 0.124167425460344, -0.168045313861751, 0.0186504943306465, -0.495880827400706, -0.206668627339351, -0.148609490335447, 0.066844529089804), GeneB_KO = c(-0.870157285953621, -0.35900872537626, -0.990406888128838, -0.120656981579292, -1.41427631791049, -0.866421095836781, -0.816936112146099, -0.894597236248507, -1.00084584207823, -0.850754488133456, -1.05314239761653, -1.07580675428372, -1.24708324665688, -1.04098460460899, -0.232388675985086, -1.48513703581865, -1.48373348870346, -1.76484058374354)), class = &quot;data.frame&quot;, row.names = c(&quot;ModelM&quot;, &quot;ModelO&quot;, &quot;ModelS&quot;, &quot;ModelA&quot;, &quot;ModelI&quot;, &quot;ModelC&quot;, &quot;ModelT&quot;, &quot;ModelH&quot;, &quot;ModelE&quot;, &quot;ModelR&quot;, &quot;ModelA2&quot;, &quot;ModelP&quot;, &quot;ModelE2&quot;, &quot;ModelU&quot;, &quot;ModelT2&quot;, &quot;ModelC2&quot;, &quot;ModelS2&quot;, &quot;ModelX&quot;))
  mut &lt;- structure(list(ModelM = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelO = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelS = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelA = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelT = c(0L, 0L, 0L, 1L, 0L, 1L, 0L, 1L, 1L, 0L), ModelH = c(1L, 1L, 1L, 1L, 0L, 1L, 1L, 1L, 0L, 0L), ModelE = c(1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L), ModelR = c(1L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 1L, 0L), ModelA2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelP = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L), ModelE2 = c(1L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 0L, 0L), ModelU = c(1L, 0L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 1L), ModelT2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI2 = c(1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC2 = c(1L, 1L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L),     ModelS2 = c(0L, 1L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 1L)), class = &quot;data.frame&quot;, row.names = c(&quot;Gene1_mut&quot;, &quot;Gene2_mut&quot;, &quot;Gene3_mut&quot;, &quot;Gene4_mut&quot;, &quot;Gene5_mut&quot;, &quot;Gene6_mut&quot;, &quot;Gene7_mut&quot;, &quot;Gene8_mut&quot;, &quot;Gene9_mut&quot;, &quot;Gene10_mut&quot;))
  </code></pre>
  <p><code>mut.csv</code> and <code>expr1.csv</code> exported for use in Python:</p>
  <pre class="lang-r prettyprint-override"><code>write.csv(mut, &quot;mut.csv&quot;, quote = FALSE, row.names = FALSE)
  write.csv(expr1[1:10,], &quot;expr1.csv&quot;, quote = FALSE, row.names = FALSE)
  </code></pre>
  <p>I want to do a one-way ANOVA in Python so that calculate P-VALUE for each pair of mutation-expression in each pair of gene-model</p>
  <p>This code returns error</p>
  <pre class="lang-py prettyprint-override"><code>import os
  import pandas as pd
  from statsmodels.formula.api import ols
  import statsmodels.api as sm
  m = pd.read_csv(&quot;mut.csv&quot;, sep=&quot;,&quot;)
  m=pd.DataFrame(m)
  m.info()
  g = pd.read_csv(&quot;expr1.csv&quot;, sep=&quot;,&quot;)
  g=pd.DataFrame(g)
  
  lst = list(g.columns)
  m_lst = list(m.columns)
  
  dfs = pd.DataFrame({})
  for Mut in m_lst:
      for Model in lst:
          df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)
          df = df.rename(columns={Mut: &quot;mut&quot;, Model: &quot;g&quot;})
          model = ols('g ~ C(mut)', data=df).fit()
  ValueError: endog has evaluated to an array with multiple columns that has shape (10, 18)
  
          aov_table = sm.stats.anova_lm(model, typ=2)
          aov_table[['Model']] = Model
          aov_table[['Mut']] = Mut
          dfs = dfs.append(aov_table)
  </code></pre>
  <p>The ideal output is something like</p>
  <pre class="lang-py prettyprint-override"><code>with pd.option_context('display.max_rows', None, 'display.max_columns', None):
      print(dfs)
              sum_sq   df         F    PR(&gt;F)     Model      Mut
  C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelM
  Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelM
  C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelM
  Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelM
  C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelO
  Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelO
  </code></pre>
  
Tags: <python><for-loop>
Title: For loop in Python goes wrong
ThreadText: # For loop in Python goes wrong
  
  (Asked by: username_0 on 2023-09-27)
  
  I have mutation status (`0`,`1`) for ten genes in ten cancer 3D models
  
  I have also gene expression in these 3D models, like
  
  ```r
  expr1 <- structure(list(GeneA_KO = c(0.0756383119639074, -0.162841965682723, -0.110703100204517, -0.457154616239399, -0.505815193885352, 0.366072775033448, 0.288650458935083, -0.0287646498640309, -0.690732824311905, 0.157366122620995, -1.29451546737684, 0.124167425460344, -0.168045313861751, 0.0186504943306465, -0.495880827400706, -0.206668627339351, -0.148609490335447, 0.066844529089804), GeneB_KO = c(-0.870157285953621, -0.35900872537626, -0.990406888128838, -0.120656981579292, -1.41427631791049, -0.866421095836781, -0.816936112146099, -0.894597236248507, -1.00084584207823, -0.850754488133456, -1.05314239761653, -1.07580675428372, -1.24708324665688, -1.04098460460899, -0.232388675985086, -1.48513703581865, -1.48373348870346, -1.76484058374354)), class = "data.frame", row.names = c("ModelM", "ModelO", "ModelS", "ModelA", "ModelI", "ModelC", "ModelT", "ModelH", "ModelE", "ModelR", "ModelA2", "ModelP", "ModelE2", "ModelU", "ModelT2", "ModelC2", "ModelS2", "ModelX"))
  mut <- structure(list(ModelM = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelO = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelS = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelA = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelT = c(0L, 0L, 0L, 1L, 0L, 1L, 0L, 1L, 1L, 0L), ModelH = c(1L, 1L, 1L, 1L, 0L, 1L, 1L, 1L, 0L, 0L), ModelE = c(1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L), ModelR = c(1L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 1L, 0L), ModelA2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelP = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L), ModelE2 = c(1L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 0L, 0L), ModelU = c(1L, 0L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 1L), ModelT2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI2 = c(1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC2 = c(1L, 1L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L),     ModelS2 = c(0L, 1L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 1L)), class = "data.frame", row.names = c("Gene1_mut", "Gene2_mut", "Gene3_mut", "Gene4_mut", "Gene5_mut", "Gene6_mut", "Gene7_mut", "Gene8_mut", "Gene9_mut", "Gene10_mut"))
  
  ```
  
  `mut.csv` and `expr1.csv` exported for use in Python:
  
  ```r
  write.csv(mut, "mut.csv", quote = FALSE, row.names = FALSE)
  write.csv(expr1[1:10,], "expr1.csv", quote = FALSE, row.names = FALSE)
  
  ```
  
  I want to do a one-way ANOVA in Python so that calculate P-VALUE for each pair of mutation-expression in each pair of gene-model
  
  This code returns error
  
  ```py
  import os
  import pandas as pd
  from statsmodels.formula.api import ols
  import statsmodels.api as sm
  m = pd.read_csv("mut.csv", sep=",")
  m=pd.DataFrame(m)
  m.info()
  g = pd.read_csv("expr1.csv", sep=",")
  g=pd.DataFrame(g)
  
  lst = list(g.columns)
  m_lst = list(m.columns)
  
  dfs = pd.DataFrame({})
  for Mut in m_lst:
      for Model in lst:
          df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)
          df = df.rename(columns={Mut: "mut", Model: "g"})
          model = ols('g ~ C(mut)', data=df).fit()
  ValueError: endog has evaluated to an array with multiple columns that has shape (10, 18)
  
          aov_table = sm.stats.anova_lm(model, typ=2)
          aov_table[['Model']] = Model
          aov_table[['Mut']] = Mut
          dfs = dfs.append(aov_table)
  
  ```
  
  The ideal output is something like
  
  ```py
  with pd.option_context('display.max_rows', None, 'display.max_columns', None):
      print(dfs)
              sum_sq   df         F    PR(>F)     Model      Mut
  C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelM
  Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelM
  C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelM
  Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelM
  C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelO
  Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelO
  
  ```
  
  **Tags:** python, for-loop
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-09-27. Score: 1
  
  I can see what you're wanting to do, I can see bugs but the key bit of output is missing (below). It seems a bit weird to me the nested loop in context, but thats fine if you're happy.
  
  The critical step is this
  
  ```
  df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)
  
  ```
  
  What I'm concerned about is the `pd.concat`. Are you sure `pd.concat` is the right command here? I thought `m` is one data table, e.g. mutation and `g` is another e.g. expression. `pd.concat` isn't going to work here, are you sure this isn't a `merge` or `join` command?
  
  Anyway, the fitting is complaining the `.shape()` is wrong. Needs reshaping? Thats the error. Printing `df` at that point would be needed.
  
  > ValueError: endog has evaluated to an array with multiple columns that has shape (10, 18)
  
  To summarise normally the error you have received is about re-shaping the data. I've not put forward that solution here because,
  
  * I am not sure what the data represents
  * If I was doing this I wouldn't use `pd.concat`
  
  It seems a lot of columns for a 1-way ANOVA, but could be right, I've not done 1-way ANOVA across 10 columns.
  
  *Bug 1*
  
  ```
  m = pd.read_csv("mut.csv", sep=",")
  m=pd.DataFrame(m)
  m.info()
  g = pd.read_csv("expr1.csv", sep=",")
  g=pd.DataFrame(g)
  
  ```
  
  This should simply be:
  
  ```
  m = pd.read_csv("mut.csv")
  g = pd.read_csv("expr1.csv")
  
  ```
  
  *Bug 2?*
  
  ```
  dfs = pd.DataFrame({})
  ...
  dfs = dfs.append(aov_table)
  
  ```
  
  This will work given `aov_table` output is a pandas dataframe (I'm not at all sure it is). You need `aov_table.info()` to be sure. You can always place it in a dataframe however.
  
  
  ---
  
  