<blockquote>
<p>I am wondering whether I can conclude that: anticollusion laws, in general, have weak but consistent negative impact on Y ceteris paribus, on average in this situation?</p>
</blockquote>
<p>I think that would be too strong wording in this case.</p>
<p>Here the results are not very consistent or robust since in the model 1 the effect is positive and in model 2-4 the effect is not statistically significant so you should there you cannot reject the null hypothesis of true effect being 0. You should not interpret results in 2-4 as saying the effect is negative (well for 2 you could at 10% level). So you can only really say there is a significant negative effect in 2 (3 at 10%) models, and even then the magnitude of the effect is not necessarily consistent.</p>
<p>However, when it comes to the magnitude of the effects, it is hard to comment on the magnitude of the coefficients since you do not say what <span class="math-container">$Y$</span> is or how is it measured, ignoring the first result with no controls, the magnitude of a coefficient in model 5 is 35,7% higher than the magnitude of the coefficient in model 6. Depending on how <span class="math-container">$Y$</span> is measured this might be high difference. For example, if <span class="math-container">$Y$</span> is output in billions that would a large difference in effect, if it is output in dollars that is difference of just few cents. Asking whether effect here is weak or strong in economic sense is essentially asking whether 0.02 is high number... well if its 0.02 billions or trillions of something then it could be a strong effect.</p>

CC BY-SA 4.0
2021-06-16T20:39:58.697Z
Hi @1muflon1, sorry for this unclear, the dependent variable here is the actual number from the ratios. For example, if dependent variable is 2%, I will let the value is 0.02 in this case and run the regression. Please let me know if it is still not clear.
Phil Nguyen
CC BY-SA 4.0
2021-06-16T20:44:50.913Z
@Knowledge-chaser ok but ratio of what if 0.02 is return  2% higher/lower ROI that is actually quite a large effect, maybe not super strong, but 2% increase/decrease in ROI is nothing to scoff at, but for some other variable 2% change might be nothing
1muflon1
CC BY-SA 4.0
2021-06-16T20:45:10.280Z
Apart from that, I follow [Dasgupta, 2019](https://academic.oup.com/rfs/article/32/7/2587/5079300) to replicate the work. It is really a mess for the first regression because it is just only controlling for $pt_{kt}$. So, I am wondering why they include this regression into the report model, is it okay to exclude it out? And what we can explain when including variables caused the change in the sign of the coefficient of variable of interest like that ?
Phil Nguyen
CC BY-SA 4.0
2021-06-16T20:51:13.773Z
I agree with you on that part of 2% change in ratio, but in the picture of the post, I saw the difference in coefficiences of $pt_{kt}$ between column (5) and (6) are $-0.0196 - (-0.0144) = - 0.0052$ , which mean 0.52%, but the column (5) is for the whole sample but column (6) is for the whole sample but excluding US, so I think it should be fine, I hope I did not fall into any fallacy
Phil Nguyen
CC BY-SA 4.0
2021-06-16T20:55:25.450Z
And, I am wondering if we can have any general conclusion for the impact of laws on the dependent Y in this case? Is it that **"anticollusion laws, in general, have insignificant on Y ceteris paribus, on average"**. Thank you so much.
Phil Nguyen
CC BY-SA 4.0
2021-06-16T21:20:12.800Z
@Knowledge-chaser people typically include multiple models to see how robust coefficients are to addition of controls, even though the most parsimonious model without any controls will likely suffer from OVB its still often done, I don't see any issue with that but also I would definitely not focus on coefficient reported in first regression. As to what explains the change in point estimates its likely the omitted variable bias (OVB) mentioned above
1muflon1
CC BY-SA 4.0
2021-06-16T21:27:00.617Z
@Knowledge-chaser please read more carefully what I have written, I did not say that there is a difference of 35 percent, I stated the coefficient has 34% higher magnitude since $-0.019/-0.014\approx 1.357$
1muflon1
CC BY-SA 4.0
2021-06-16T21:30:37.323Z
@Knowledge-chaser no I do not think that would be correct interpretion as well here too.  What I would have written here is that, the effect of the anti-collision laws is not robust. Most general models report statistically significant negative effect but coefficient magnitude is not robust either (this last part after but still depends on what even Y is since you did not mentioned it in the comments). More parsimonious models are either not significant or report significant positive effect. After that you can add some arguments why you think that higher weight should be put on last 2 results
1muflon1
CC BY-SA 4.0
2021-06-16T21:56:48.887Z
They are great answers, I learnt a lot, thank you @1muflon1
Phil Nguyen
CC BY-SA 4.0
2021-06-16T14:36:28.020Z
2021-06-16T21:45:08.410Z
2021-06-16T21:45:08.410Z
1muflon1
<p>I saw the impact of anticollusion laws on dependent variables Y across the country by using generalized DID by following <a href="https://academic.oup.com/rfs/article/32/7/2587/5079300" rel="nofollow noreferrer">Dasgupta, 2019</a>.</p>
<p>The identification is:</p>
<p><span class="math-container">$Y_{it}$</span> = <span class="math-container">$\alpha$</span> + <span class="math-container">$\beta$</span> <span class="math-container">$(pt)_{kt}$</span> + <span class="math-container">$\delta$$X_{ikt}$</span> + <span class="math-container">$\theta$$_t$</span> + <span class="math-container">$\gamma$$_i$</span> +<span class="math-container">$\epsilon$$_{it}$</span></p>
<p>where <em>i</em>,<em>k</em>, and <em>t</em> index firms, countries, and years respectively. <span class="math-container">$X_{ikt}$</span> is a vector of the different firm, country, and industry control, while <span class="math-container">$\gamma$</span> and <span class="math-container">$\theta$</span> are firm and year fixed effects.<span class="math-container">$(pt)_{kt}$</span> is the post * treat variable</p>
<p>The result is</p>
<p><a href="https://i.sstatic.net/8PuZ9.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8PuZ9.png" alt="enter image description here" /></a></p>
<p>While 6 columns all using firm and years fixed effects if not stated elsewhere.Column 1, I did not control any independent variables. Column(2), I control for some firm and country independent variables. In column (3), I control for firm, country and industry variables. in column (4), I control for country and firm independent variable along with firm and industry * year fixed effect. In column (5). I control for the country and firm independent variables along with firm and region * year fixed effect. In column (6), I control for some firm and country independent variables, similar to column (2) but without US firms.</p>
<p>I am wondering whether I can conclude that: <strong>anticollusion laws, in general, have weak but consistent negative impact on Y ceteris paribus, on average</strong> in this situation?</p>

<econometrics><treatment-effect>
What should we conclude when the significant levels are different but the sign are similar among coefficients of variable of interest?
Phil Nguyen
# What should we conclude when the significant levels are different but the sign are similar among coefficients of variable of interest?

(Asked by: username_0 on 2021-06-16)

I saw the impact of anticollusion laws on dependent variables Y across the country by using generalized DID by following [Dasgupta, 2019](https://academic.oup.com/rfs/article/32/7/2587/5079300).

The identification is:

$Y\_{it}$ = $\alpha$ + $\beta$ $(pt)\_{kt}$ + $\delta$$X\_{ikt}$ + $\theta$$\_t$ + $\gamma$$\_i$ +$\epsilon$$\_{it}$

where *i*,*k*, and *t* index firms, countries, and years respectively. $X\_{ikt}$ is a vector of the different firm, country, and industry control, while $\gamma$ and $\theta$ are firm and year fixed effects.$(pt)\_{kt}$ is the post \* treat variable

The result is

[![enter image description here](https://i.sstatic.net/8PuZ9.png)](https://i.sstatic.net/8PuZ9.png)

While 6 columns all using firm and years fixed effects if not stated elsewhere.Column 1, I did not control any independent variables. Column(2), I control for some firm and country independent variables. In column (3), I control for firm, country and industry variables. in column (4), I control for country and firm independent variable along with firm and industry \* year fixed effect. In column (5). I control for the country and firm independent variables along with firm and region \* year fixed effect. In column (6), I control for some firm and country independent variables, similar to column (2) but without US firms.

I am wondering whether I can conclude that: **anticollusion laws, in general, have weak but consistent negative impact on Y ceteris paribus, on average** in this situation?

**Tags:** econometrics, treatment-effect


---

## 1 answers

### Answer by username_1 on 2021-06-16. Score: 1 | Accepted answer

> I am wondering whether I can conclude that: anticollusion laws, in general, have weak but consistent negative impact on Y ceteris paribus, on average in this situation?

I think that would be too strong wording in this case.

Here the results are not very consistent or robust since in the model 1 the effect is positive and in model 2-4 the effect is not statistically significant so you should there you cannot reject the null hypothesis of true effect being 0. You should not interpret results in 2-4 as saying the effect is negative (well for 2 you could at 10% level). So you can only really say there is a significant negative effect in 2 (3 at 10%) models, and even then the magnitude of the effect is not necessarily consistent.

However, when it comes to the magnitude of the effects, it is hard to comment on the magnitude of the coefficients since you do not say what $Y$ is or how is it measured, ignoring the first result with no controls, the magnitude of a coefficient in model 5 is 35,7% higher than the magnitude of the coefficient in model 6. Depending on how $Y$ is measured this might be high difference. For example, if $Y$ is output in billions that would a large difference in effect, if it is output in dollars that is difference of just few cents. Asking whether effect here is weak or strong in economic sense is essentially asking whether 0.02 is high number... well if its 0.02 billions or trillions of something then it could be a strong effect.

> **Comments:**
> **username_0**: Hi @username_1, sorry for this unclear, the dependent variable here is the actual number from the ratios. For example, if dependent variable is 2%, I will let the value is 0.02 in this case and run the regression. Please let me know if it is still not clear.  
> **username_1**: @Knowledge-chaser ok but ratio of what if 0.02 is return  2% higher/lower ROI that is actually quite a large effect, maybe not super strong, but 2% increase/decrease in ROI is nothing to scoff at, but for some other variable 2% change might be nothing  
> **username_0**: Apart from that, I follow [Dasgupta, 2019](https://academic.oup.com/rfs/article/32/7/2587/5079300) to replicate the work. It is really a mess for the first regression because it is just only controlling for $pt_{kt}$. So, I am wondering why they include this regression into the report model, is it okay to exclude it out? And what we can explain when including variables caused the change in the sign of the coefficient of variable of interest like that ?  
> **username_0**: I agree with you on that part of 2% change in ratio, but in the picture of the post, I saw the difference in coefficiences of $pt_{kt}$ between column (5) and (6) are $-0.0196 - (-0.0144) = - 0.0052$ , which mean 0.52%, but the column (5) is for the whole sample but column (6) is for the whole sample but excluding US, so I think it should be fine, I hope I did not fall into any fallacy  
> **username_0**: And, I am wondering if we can have any general conclusion for the impact of laws on the dependent Y in this case? Is it that **"anticollusion laws, in general, have insignificant on Y ceteris paribus, on average"**. Thank you so much.  
> **username_1**: @Knowledge-chaser people typically include multiple models to see how robust coefficients are to addition of controls, even though the most parsimonious model without any controls will likely suffer from OVB its still often done, I don't see any issue with that but also I would definitely not focus on coefficient reported in first regression. As to what explains the change in point estimates its likely the omitted variable bias (OVB) mentioned above  
> **username_1**: @Knowledge-chaser please read more carefully what I have written, I did not say that there is a difference of 35 percent, I stated the coefficient has 34% higher magnitude since $-0.019/-0.014\approx 1.357$  
> **username_1**: @Knowledge-chaser no I do not think that would be correct interpretion as well here too.  What I would have written here is that, the effect of the anti-collision laws is not robust. Most general models report statistically significant negative effect but coefficient magnitude is not robust either (this last part after but still depends on what even Y is since you did not mentioned it in the comments). More parsimonious models are either not significant or report significant positive effect. After that you can add some arguments why you think that higher weight should be put on last 2 results  
> **username_0**: They are great answers, I learnt a lot, thank you @username_1  

---

