Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
| From | Joerg Luedicke <joerg.luedicke@gmail.com> |
| To | statalist@hsphsun2.harvard.edu |
| Subject | Re: st: Cross-level interaction with xtmixed |
| Date | Tue, 10 Jan 2012 14:35:39 -0500 |
In the model with the interaction effects, the coefficient for indepvar1 is the effect for indepvar1 when indepvar2 is zero, the coefficient for indepvar2 is the effect for indepvar2 when indepvar1 is zero (so, it is not reasonable to say that "the effect of indepvar1 becomes insignificant", or something to that effect, because you cannot compare this coefficient with the one from the model without the interaction term). The interaction term says that the effect of indepvar1 decreases by .030349 for a unit change in indepvar2. That is what you also observe with the marginal effects (e.g. 0.1260579-0.095709=.0303489 etc.). What seems to be important now is to check whether these are meaningful differences with respect to substantial matters. That is, is the main effect of 0.07 a meaningful measure and would the variation of this measure across levels of the other variable imply anything meaningful? I would worry about "significance testing" later. One problem with the latter in your current approach is that you only have 12 higher level units, which is on the very low end and might not be sufficient for variance estimation and testing of varying coefficients. Have you looked at separate regressions (by the 12 higher level units) and eyeballed differences in indepvar1 across groups? I would start with that and see where that leads. HTH, J. On Tue, Jan 10, 2012 at 12:30 PM, Marc Peters <marcpeters32c@yahoo.se> wrote: > Thank you Joerg, > > > Of course, Here are the outputs. > > > *MODEL 1, MAIN MODEL WITH RANDOM INTERCEPTS > > > > . xtmixed depvar indepvar1 indepvar2 indepvar3 indepvar4 ||id:, var > > Performing EM optimization: > > Performing gradient-based optimization: > > Iteration 0: log likelihood = 1403.7234 > Iteration 1: log likelihood = 1403.7234 > > Computing standard errors: > > Mixed-effects ML regression Number of obs = 2030 > Group variable: id Number of groups = 12 > > Obs per group: min = 136 > avg = 169.2 > max = 223 > > > Wald chi2(4) = 36.76 > Log likelihood = 1403.7234 Prob > chi2 = 0.0000 > > ------------------------------------------------------------------------------ > depvar | Coef. Std. Err. z P>|z| [95% Conf. Interval] > -------------+---------------------------------------------------------------- > indepvar1 | .0756643 .0146331 5.17 0.000 .046984 .1043447 > indepvar2 | .0374981 .0275734 1.36 0.174 -.0165447 .091541 > indepvar3 | .0274294 .013366 2.05 0.040 .0012325 .0536264 > indepvar4 | -.023015 .0162651 -1.41 0.157 -.0548941 .0088641 > _cons | .1835104 .0813048 2.26 0.024 .024156 .3428648 > ------------------------------------------------------------------------------ > > > *MODEL 2, WITH INTERACTION > > > > . xtmixed depvar c.indepvar1##c.indepvar2 indepvar3 indepvar4 ||id: indepvar1, var cov(un) > > Performing EM optimization: > > Performing gradient-based optimization: > > Iteration 0: log likelihood = 1471.8551 > Iteration 1: log likelihood = 1471.8551 > > Computing standard errors: > > Mixed-effects ML regression Number of obs = 2030 > Group variable: id Number of groups = 12 > > Obs per group: min = 136 > avg = 169.2 > max = 223 > > > Wald chi2(5) = 8.32 > Log likelihood = 1471.8551 Prob > chi2 = 0.1394 > > ----------------------------------------------------------------------------------------- > depvar | Coef. Std. Err. z P>|z| [95% Conf. Interval] > ------------------------+---------------------------------------------------------------- > indepvar1 | .1564069 .1446754 1.08 0.280 -.1271518 .4399656 > indepvar2 | .0379993 .0504303 0.75 0.451 -.0608422 .1368408 > | > c.indepvar1#c.indepvar2 | -.030349 .0597786 -0.51 0.612 -.1475128 .0868149 > | > indepvar3 | .0277277 .0130601 2.12 0.034 .0021305 .053325 > indepvar4 | .004068 .0164534 0.25 0.805 -.0281802 .0363161 > _cons | .0286898 .130898 0.22 0.827 -.2278655 .2852452 > ----------------------------------------------------------------------------------------- > > ------------------------------------------------------------------------------ > Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval] > -----------------------------+------------------------------------------------ > id: Unstructured | > var(indepv~1) | .0333422 .0147683 .0139949 .0794363 > var(_cons) | .0217169 .00978 .0089839 .0524964 > cov(indepv~1,_cons) | -.0246262 .0114631 -.0470934 -.0021589 > -----------------------------+------------------------------------------------ > var(Residual) | .013216 .0004171 .0124232 .0140594 > ------------------------------------------------------------------------------ > LR test vs. linear regression: chi2(3) = 495.38 Prob > chi2 = 0.0000 > > Note: LR test is conservative and provided only for reference. > > > > *MARGINAL EFFECTS > > > . margins, dydx(indepvar1) at (indepvar2=(1(1) 4)) vsquish post > > Average marginal effects Number of obs = 2030 > > Expression : Linear prediction, fixed portion, predict() > dy/dx w.r.t. : indepvar1 > 1._at : indepvar2 = 1 > 2._at : indepvar2 = 2 > 3._at : indepvar2 = 3 > 4._at : indepvar2 = 4 > > ------------------------------------------------------------------------------ > | Delta-method > | dy/dx Std. Err. z P>|z| [95% Conf. Interval] > -------------+---------------------------------------------------------------- > indepvar1 | > _at | > 1 | .1260579 .0922453 1.37 0.172 -.0547396 .3068555 > 2 | .095709 .0568716 1.68 0.092 -.0157572 .2071752 > 3 | .06536 .0714598 0.91 0.360 -.0746986 .2054186 > 4 | .035011 .1188511 0.29 0.768 -.1979328 .2679549 > ------------------------------------------------------------------------------ > > > > Thank you so much for your help! > > > Best, > > Marc > > > > ----- Ursprungligt meddelande ---- > Från: Joerg Luedicke <joerg.luedicke@gmail.com> > Till: statalist@hsphsun2.harvard.edu > Kopia: > Skickat: tisdag, 10 januari 2012 16:57 > Ämne: Re: st: Cross-level interaction with xtmixed > > Perhaps it would be helpful if you could post the output of the two > models plus output from your marginal effects calculations. > > J. > > On Tue, Jan 10, 2012 at 7:34 AM, Marc Peters <marcpeters32c@yahoo.se> wrote: >> I really don't want to bug you with this question. But if anyone know anything about this I would appreciate your help tremendously. >> Please don't hesitate to ask me if you need more information. >> >> Best, >> >> Marc >> >> >> ----- Ursprungligt meddelande ---- >> Från: Marc Peters <marcpeters32c@yahoo.se> >> Till: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu> >> Kopia: >> Skickat: måndag, 9 januari 2012 10:20 >> Ämne: st: Cross-level interaction with xtmixed >> >> Dear all, >> >> I am trying to run a multi-level OLS-regression with a cross-level interaction. My main model uses random intercepts (but not random slopes). >> >> I use the xtmixed command in Stata: >> >> xtmixed depvar indepvar1 indepvar2 indepvar3 indepvar|| groupvar:, var >> >> In this model indepvar1 (continuous level-1 variable) becomes highly significant. Indepvar1 is the only level-1 variable in the model. I would like to see whether the effect of indepvar1 decreases if interacted with indepvar2 (a continuous level-2 variable with only four observed values). Since this is a cross-level interaction I need to sepcify a random slope model: >> >> xtmixed depvar indepvar1 indepvar2 indepvar1*indepvar2 indepvar3 indepvar4|| groupvar: indepvar1, cov(un) >> >> In this model both indepvar1 and indepvar1*indepvar2 becomes insignificant. When using the margins command to see the marginal effect of indepvar1 at the different levels of indepvar2, indepvar1 is insignificant at every level of indepvar2. I have a porblem understanding why this is or how to interpret these results. >> >> Rerunning model 1 with random slopes (but with no interactions) indepvar1 also gets insignificant. Is there a problem with how I have specified the models or how should I understand the results? Why is a variable insignificant with random slopes, but significant without them? >> >> Best, >> >> Marc >> >> >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/statalist/faq >> * http://www.ats.ucla.edu/stat/stata/ >> >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/statalist/faq >> * http://www.ats.ucla.edu/stat/stata/ > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/