Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down on April 23, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
Maarten Buis <maartenlbuis@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: What does "The effect is significant" after xtprobit mean:examining ALL three effects? |

Date |
Tue, 21 Feb 2012 10:17:34 +0100 |

On Tue, Feb 21, 2012 at 6:29 AM, Amanda Fu wrote: > I am using an probit random effect estimation (-xtprobit) to examine > the effect of a key variable (an income shock) on some outcome (work > or notwork). I want to report if the effect is significant or not. > > Previously statalisters had discussed about this topic. From the > following link there is a great summary by Mr. Maarten Buis. > http://www.stata.com/statalist/archive/2008-10/msg01122.html (Thank > you, Mr Buis!) > > What I am not sure is the last sentence in Mr. Buis's suggestion (see > below). Does Mr. Buis suggest that when we claim that an effect is > significant, we usually examine ALL the following three effects and > ALL of them must reject the null hypotheses? > (A) the effect on the latent propensity (-probit-), > (B) the effect on the probability for someone with typical values on > the explanatory variables > (-mfx- or -margeff- with the -at(mean)- option), > (C) the effect on the probability for a typical person (-margeff- > without the -at(mean)- option). > > What if only two of these effects are significant? My results are that > only (A) are significant. Significance refers to a statistical test, and a statistical test refers to a null hypothesis. A null hypothesis is just one statement, and the aim of the test is to see whether your data is so inconsistent with that statement that you can reject the statement. When that happens we sometimes get lazy and just say that the parameter is significant. Apparently this short-hand confused you (it is a very common form of statistical confusion), so the best way is not to be lazy and spell out the complete sequence, that is, explicitly state the null-hypothesis, which means you have to choose one of your three options, state the test statistic and the decision whether or not to reject the null hypothesis. So now the question becomes, which of the options A, B, or C do you choose? If you want "the effect" of a variable, that is, the effect in the form of one number and not in the form of multiple numbers/a graph, than you should not use B or C. In essence a marginal effect tells you how much you expect your probability to increase for a unit chock in wage. If you report only one such an effect you are in essence converting your probit model in a linear probability model. If that is what you want than you should do so directly and check whether the linearity assumption is reasonable for your data. So if you really want to report just one additive effect for each variable, than I would use -xtreg- instead of -xtprobit- with marginal effects. Whether or not option A is attractive is a matter of taste (your own and your discipline's). In my and my discipline's taste it is not very attractive, but tastes notoriously differ. I would also at least consider -xtlogit- and look at the odds ratios or -xtpoisson- and look at the risk ratios. For -xtpoisson- you need to check, just like the linear probability model, whether you get too many implausible predictions. The common theme with these recommendations is that you first choose the kind of effect you want. An effect is just a comparison of groups; you can look at differences in probabilities between person one unit wage shock apart, or differences in latent propensity between persons one unit wage shock apart, or at ratios of probabilities between persons one unit wage shock apart, or at ratios of odds between persons one unit wage shock apart. Once you have chosen your effect you choose the model which directly gives you the effect of your choice. There has been a long discussion on this in the thread starting with <http://www.stata.com/statalist/archive/2012-02/msg00256.html>. In particular there has been some skepticism about the value of the linear probability model. Some of that skepticism has been, in my opinion, too strong, but I agree that there are situations where the linear probability model is not unproblematic. My only claim is that reporting one marginal effect per variable is worse than a linear probability model. So if one has problems with the linear probability model, than estimating a non-linear model and reporting marginal effects is certainly not the solution. Instead one should in that situation rethink the kind of effect one is interested in. > Additionally, I notice most published papers just say "Marginal > effects of probit are reported". Which effect do most people mean when > they simply say "marginal effects"? That depends to a large extend to when those papers where published and in what discipline. So to answer that question you really need to know the state of that (sub-(sub-))discipline at the point that those articles were written. Hope this helps, Maarten -------------------------- Maarten L. Buis Institut fuer Soziologie Universitaet Tuebingen Wilhelmstrasse 36 72074 Tuebingen Germany http://www.maartenbuis.nl -------------------------- * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**st: What does "The effect is significant" after xtprobit mean:examining ALL three effects?***From:*Amanda Fu <mandy.fu1@gmail.com>

**References**:**st: What does "The effect is significant" after xtprobit mean:examining ALL three effects?***From:*Amanda Fu <mandy.fu1@gmail.com>

- Prev by Date:
**st: scale parameter in glm** - Next by Date:
**st: update margdistfit** - Previous by thread:
**st: What does "The effect is significant" after xtprobit mean:examining ALL three effects?** - Next by thread:
**st: What does "The effect is significant" after xtprobit mean:examining ALL three effects?** - Index(es):