Answers.Body: <p>Second viewing of the question from what I can see -0.22 as a coefficient of origin is a strong negative association, so yeah it has a major impact. Its not how I would have done it, but that looks to be the result.</p>
  <hr />
  <p>First viewing,</p>
  <p>I'm going to throw my hat in here. We don't know what 'origin' is about, anyway just throw everything, i.e. each substrate and the origin into the same regression calculation. Check for a low-residual and preferably do a Q-Q plot, transform your data it this doesn't look good.</p>
  <p>The key and the thing you are missing is your regression weights, without that I couldn't say very much. If the regression weight is near zero for 'origin' then it has zero impact. If the regression weight of 'origin' is positively greater than everything else ... I assume there are skewed distributions of 'substrates' between the 'origins'. If the regression weight of 'origin' is negative but still greater than all other regression weights then it is adversely affecting the 'value' you are seeking.</p>
  <p>I don't know the experiment, the biological system or really the 'substrate' assays, so I can't comment any further.</p>
  <p>The two issues I have are:</p>
  <ol>
  <li>Doing an ANOVA on the output of a regression analysis doesn't make much sense to me. It is not something I would do, nor something in ML or GLM I've encountered.</li>
  <li>Are you doing pairwise substrate/origin calculations? I presue not, but just in case this not how GLM works.</li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:31:18.023Z
Answers.Text: oh I was under the impression `aov(summary(glm_model))` was a pretty common practice as it nicely summarises down to a variable level? Where do I find the regression weights?
Answers.UserDisplayName: Lamma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:36:41.693Z
Answers.Text: When transforming the data do I have to transform both variables or can I transform just one of them until they fit the same distribution? and given that I am using a negative binomial glm does a uniform distribution between the variables matter as much?
  
  I am unsure if there is a skewed distribution of variables between origins, that is what I am looking to find out really.
Answers.UserDisplayName: Lamma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:36:45.813Z
Answers.Text: hi @Michael, very nice points here, yes the term anova is quite misleading sometimes. Normally people associate associate with the classical analysis of variance. In more "modern" uses, it can refer to the breakdown of deviance in a model, https://stats.stackexchange.com/questions/289226/what-are-the-differences-between-anovas-and-glms
Answers.UserDisplayName: StupidWolf
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:38:27.387Z
Answers.Text: It is like a log likelihood ratio test essentially, the "F" statistic instead of Chi-sq is used.
Answers.UserDisplayName: StupidWolf
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:39:58.853Z
Answers.Text: @StupidWolf So it is fine to `aov(summary(glm_model))` to get a nice overview of the model? And by pairwise substrate/origin calculations does you mean doing the model on a subsatrate of the dataframe for each substrate @michael?
Answers.UserDisplayName: Lamma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-27T12:46:40.800Z
Answers.LastActivityDate: 2020-06-29T00:35:18.887Z
Answers.LastEditDate: 2020-06-29T00:35:18.887Z
Answers.UserDisplayName: M__
Answers.Body: <p>There is no better method, it's a matter of what you want to test or what is your question.</p>
  <p>Using the <code>anova()</code> or <code>aov()</code>, test the terms collectively. For example, in your example with Substrate, the null hypothesis is that the coefficients are all zero, meaning <code>cellulose =0, mannan =0 , ....</code></p>
  <p>If the question is, &quot;do the isolate samples have a higher value than origin samples?&quot;, then you can use your first model, where <code>free</code> is set as the reference and you test whether the effect of <code>isolate</code> is non-zero. Likewise you can do this for substrate and set of them as your reference. You can also do other pairwise comparisons using this model.</p>
  <p>If the question is, &quot;does origin have a significant effect on value, after controlling for substrate?&quot;, then you can use your second model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:20:25.070Z
Answers.Text: Would I control for substrate by forming the model as such: value ~ origin + substrate ?
Answers.UserDisplayName: Lamma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:21:56.630Z
Answers.Text: yes you do. you are giving every substrate a meaning  and on top of that estimating the effect of origin
Answers.UserDisplayName: StupidWolf
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:24:53.787Z
Answers.Text: And does the order in which I give the variable to the model, i.e. origin before substrate, indicate some sort of predefined importance to the model?
Answers.UserDisplayName: Lamma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:28:18.840Z
Answers.Text: it doesn't define the importance, you can put it in any order. However if they are correlated, most of the variance will go to the first. I don't think this will apply in your case
Answers.UserDisplayName: StupidWolf
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:32:26.997Z
Answers.Text: see https://cran.r-project.org/doc/contrib/Faraway-PRA.pdf blocking design. A good ieda to read this
Answers.UserDisplayName: StupidWolf
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-29T07:33:01.713Z
Answers.Text: Thank you I will look at this :)
Answers.UserDisplayName: Lamma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-27T12:06:33.487Z
Answers.LastActivityDate: 2020-06-27T12:06:33.487Z
Answers.UserDisplayName: StupidWolf
Body: <p>How would one determine the significance of a variable in a glm model?</p>
  <p>If I, for example, have a dataframe like the one seen below, how would I determine if the origin of the sample has a significant effect on the value? (this is the number of enzymes capable of degrading the substrate of that matter)</p>
  <pre><code>Substrate    variable value origin
  cellulose       M09    8    free
  mannan          M12    2    free
  glycogen        M65    2    free
  chitin          M87    4    free
  cellulose       M90    2    isolate
  manan           M78    1    isolate
  glycogen        M21    4    isolate
  chitin          M21    1    isolate
  </code></pre>
  <p>So far I have tried:</p>
  <pre class="lang-r prettyprint-override"><code>mcomp = glm.nb(value ~ origin, data = my_data)
  
  summary(mcomp)
  Deviance Residuals: 
      Min       1Q   Median       3Q      Max  
  -0.9625  -0.9047  -0.9047   0.1212   3.5232  
  
  Coefficients:
                Estimate Std. Error z value Pr(&gt;|z|)   
  (Intercept)   -0.01657    0.06571  -0.252  0.80097   
  originisolate -0.21911    0.08180  -2.679  0.00739 **
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
  
  (Dispersion parameter for Negative Binomial(0.3418) family taken to be 1)
  
      Null deviance: 2053.5  on 2679  degrees of freedom
  Residual deviance: 2046.3  on 2678  degrees of freedom
  AIC: 6517.5
  
  Number of Fisher Scoring iterations: 1
  
  
                Theta:  0.3418 
            Std. Err.:  0.0186 
  
   2 x log-likelihood:  -6511.4590 
  </code></pre>
  <p>So free becomes the intercept and then isolate if significantly different from that. Does this mean Origin has a significant effect on the value?</p>
  <p>Would the better approach be to do the following?:</p>
  <pre class="lang-r prettyprint-override"><code>mcomp = glm.nb(value ~ origin + Substrate, data = comb_data) 
  summary(aov(mcomp))
                Df Sum Sq Mean Sq F value Pr(&gt;F)    
  origin         1     23   22.55   6.612 0.0102 *  
  Substrate     44   1445   32.84   9.631 &lt;2e-16 ***
  Residuals   2634   8981    3.41                   
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 
  </code></pre>
  <p>This shows me that origin and substrate have an effect on value if I understand correctly?</p>
  
Tags: <modelling><linear-regression><statsmodels>
Title: Determining significance of a variable in a glm model
ThreadText: # Determining significance of a variable in a glm model
  
  (Asked by: username_0 on 2020-06-26)
  
  How would one determine the significance of a variable in a glm model?
  
  If I, for example, have a dataframe like the one seen below, how would I determine if the origin of the sample has a significant effect on the value? (this is the number of enzymes capable of degrading the substrate of that matter)
  
  ```
  Substrate    variable value origin
  cellulose       M09    8    free
  mannan          M12    2    free
  glycogen        M65    2    free
  chitin          M87    4    free
  cellulose       M90    2    isolate
  manan           M78    1    isolate
  glycogen        M21    4    isolate
  chitin          M21    1    isolate
  
  ```
  
  So far I have tried:
  
  ```r
  mcomp = glm.nb(value ~ origin, data = my_data)
  
  summary(mcomp)
  Deviance Residuals: 
      Min       1Q   Median       3Q      Max  
  -0.9625  -0.9047  -0.9047   0.1212   3.5232  
  
  Coefficients:
                Estimate Std. Error z value Pr(>|z|)   
  (Intercept)   -0.01657    0.06571  -0.252  0.80097   
  originisolate -0.21911    0.08180  -2.679  0.00739 **
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
  
  (Dispersion parameter for Negative Binomial(0.3418) family taken to be 1)
  
      Null deviance: 2053.5  on 2679  degrees of freedom
  Residual deviance: 2046.3  on 2678  degrees of freedom
  AIC: 6517.5
  
  Number of Fisher Scoring iterations: 1
  
                Theta:  0.3418 
            Std. Err.:  0.0186 
  
   2 x log-likelihood:  -6511.4590 
  
  ```
  
  So free becomes the intercept and then isolate if significantly different from that. Does this mean Origin has a significant effect on the value?
  
  Would the better approach be to do the following?:
  
  ```r
  mcomp = glm.nb(value ~ origin + Substrate, data = comb_data) 
  summary(aov(mcomp))
                Df Sum Sq Mean Sq F value Pr(>F)    
  origin         1     23   22.55   6.612 0.0102 *  
  Substrate     44   1445   32.84   9.631 <2e-16 ***
  Residuals   2634   8981    3.41                   
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 
  
  ```
  
  This shows me that origin and substrate have an effect on value if I understand correctly?
  
  **Tags:** modelling, linear-regression, statsmodels
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-06-27. Score: 1
  
  Second viewing of the question from what I can see -0.22 as a coefficient of origin is a strong negative association, so yeah it has a major impact. Its not how I would have done it, but that looks to be the result.
  
  ---
  
  First viewing,
  
  I'm going to throw my hat in here. We don't know what 'origin' is about, anyway just throw everything, i.e. each substrate and the origin into the same regression calculation. Check for a low-residual and preferably do a Q-Q plot, transform your data it this doesn't look good.
  
  The key and the thing you are missing is your regression weights, without that I couldn't say very much. If the regression weight is near zero for 'origin' then it has zero impact. If the regression weight of 'origin' is positively greater than everything else ... I assume there are skewed distributions of 'substrates' between the 'origins'. If the regression weight of 'origin' is negative but still greater than all other regression weights then it is adversely affecting the 'value' you are seeking.
  
  I don't know the experiment, the biological system or really the 'substrate' assays, so I can't comment any further.
  
  The two issues I have are:
  
  1. Doing an ANOVA on the output of a regression analysis doesn't make much sense to me. It is not something I would do, nor something in ML or GLM I've encountered.
  2. Are you doing pairwise substrate/origin calculations? I presue not, but just in case this not how GLM works.
  
  > **Comments:**
  > **username_0**: oh I was under the impression `aov(summary(glm_model))` was a pretty common practice as it nicely summarises down to a variable level? Where do I find the regression weights?  
  > **username_0**: When transforming the data do I have to transform both variables or can I transform just one of them until they fit the same distribution? and given that I am using a negative binomial glm does a uniform distribution between the variables matter as much?  I am unsure if there is a skewed distribution of variables between origins, that is what I am looking to find out really.  
  > **username_2**: hi @Michael, very nice points here, yes the term anova is quite misleading sometimes. Normally people associate associate with the classical analysis of variance. In more "modern" uses, it can refer to the breakdown of deviance in a model, https://stats.stackexchange.com/questions/289226/what-are-the-differences-between-anovas-and-glms  
  > **username_2**: It is like a log likelihood ratio test essentially, the "F" statistic instead of Chi-sq is used.  
  > **username_0**: @username_2 So it is fine to `aov(summary(glm_model))` to get a nice overview of the model? And by pairwise substrate/origin calculations does you mean doing the model on a subsatrate of the dataframe for each substrate @michael?  
  
  ---
  
  ### Answer by username_2 on 2020-06-27. Score: 0
  
  There is no better method, it's a matter of what you want to test or what is your question.
  
  Using the `anova()` or `aov()`, test the terms collectively. For example, in your example with Substrate, the null hypothesis is that the coefficients are all zero, meaning `cellulose =0, mannan =0 , ....`
  
  If the question is, "do the isolate samples have a higher value than origin samples?", then you can use your first model, where `free` is set as the reference and you test whether the effect of `isolate` is non-zero. Likewise you can do this for substrate and set of them as your reference. You can also do other pairwise comparisons using this model.
  
  If the question is, "does origin have a significant effect on value, after controlling for substrate?", then you can use your second model.
  
  > **Comments:**
  > **username_0**: Would I control for substrate by forming the model as such: value ~ origin + substrate ?  
  > **username_2**: yes you do. you are giving every substrate a meaning  and on top of that estimating the effect of origin  
  > **username_0**: And does the order in which I give the variable to the model, i.e. origin before substrate, indicate some sort of predefined importance to the model?  
  > **username_2**: it doesn't define the importance, you can put it in any order. However if they are correlated, most of the variance will go to the first. I don't think this will apply in your case  
  > **username_2**: see https://cran.r-project.org/doc/contrib/Faraway-PRA.pdf blocking design. A good ieda to read this  
  > **username_0**: Thank you I will look at this :)  
  
  ---
  
  