Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
| From | Nick Cox <njcoxstata@gmail.com> |
| To | "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu> |
| Subject | Re: st: Query.. |
| Date | Wed, 17 Apr 2013 11:37:15 +0100 |
You are simulating well-behaved data -- I see -rnormal()- everywhere -- but the doubts raised are about how well tests perform in non-standard situations. Nick njcoxstata@gmail.com On 17 April 2013 11:28, John Antonakis <John.Antonakis@unil.ch> wrote: > OK....fine with this, but it has no bearing whatsoever on what I said below, > which was on overidentification and how it is tested in SEM. The chi-square > statistic of SEM will be very similar to the overidentification statistic > from a 2sls or 3sls model, and that is the point I was trying to get across. > So, if economists (and others) trust the Hansen-Sargan overidentification > statistic, then they should trust and sem chi-square overidentification > statistic (and not indexes that are not tests). > > Run this code to see that we can about the same chi-square value whether > using 2sls or sem, even though they go about it in very different way (i.e., > in 2sls, chi = r-square*N from a regression of the residuals of the y > equation on the excluded in instruments, whereas the chi-square test from > SEM uses the discrepancy function I showed below from sigmal and S): > > clear > set seed 123 > set obs 1000 > > gen x1 = rnormal() > gen x2 = rnormal() > gen q = rnormal() > gen m = x1 + x2 - q + rnormal() > gen y = m + q + rnormal() > > qui: ivregress 2sls y (m = x1 x2) > qui: estat overid > scalar chi_sargan = r(sargan) > scalar p_sargan = r(p_sargan) > > qui: sem (y<-m) (m<-x1 x2), cov(e.y*e.m) > qui: estat gof > scalar chi_sem = r(chi2_ms) > scalar p_sem = r(p_ms) > > dis "Chi Sargan = "chi_sargan ", p-value = "p_sargan > dis "Chi SEM = "chi_sem ", p-value = "p_sem > > Now, having latent variable in there does not change the basis of how this > chi-square statistic is calculated in SEM. Also, if we have one of the > conditions that makes the chi-square misbehave (that I identified below), > then we can rescale the SEM chi-square using one of the corrective > procedures so that it approximates the expected chi-square distribution. > > > Best, > J. > > > __________________________________________ > > John Antonakis > Professor of Organizational Behavior > Director, Ph.D. Program in Management > > Faculty of Business and Economics > University of Lausanne > Internef #618 > CH-1015 Lausanne-Dorigny > Switzerland > Tel ++41 (0)21 692-3438 > Fax ++41 (0)21 692-3305 > http://www.hec.unil.ch/people/jantonakis > > Associate Editor > The Leadership Quarterly > __________________________________________ > > On 17.04.2013 01:01, Lachenbruch, Peter wrote: >> >> The context i was referring to was an old article by George Box in >> Biometrika aboutg 1953 in which he commented that testing for >> heteroskedasticy was like setting to see in a rowboat to see if it was safe >> for the Queen Mary to sail. Sorry i don't have the quote, and my books are >> all bundled up due to a flood in my basement >> >> Peter A. Lachenbruch, >> Professor (retired) >> ________________________________________ >> From: owner-statalist@hsphsun2.harvard.edu >> [owner-statalist@hsphsun2.harvard.edu] on behalf of John Antonakis >> [John.Antonakis@unil.ch] >> Sent: Tuesday, April 16, 2013 1:47 PM >> To: statalist@hsphsun2.harvard.edu >> Subject: Re: st: Query.. >> >> Hello Peter: >> >> Can you please elaborate? The chi-square test of fit--or the likelihood >> ratio test comparing the saturated to the target model--is pretty >> robust, though as I indicated, it does not behave as expected at small >> samples, when data are not multivariate normal, when the model is >> complex (and the n to parameters estimated ration is low). However, as I >> mentioned there are remedies to the problem. More specifically see: >> >> Bollen, K. A., & Stine, R. A. (1992). Bootstrapping goodness-of-fit >> measures in structural equation models. Sociological Methods & Research, >> 21(2), 205-229. >> >> Herzog, W., & Boomsma, W. (2009). Small-sample robust estimators of >> noncentrality-based and incremental model fit. Structural Equation >> Modeling, 16(1), 1–27. >> >> Swain, A. J. (1975). Analysis of parametric structures for variance >> matrices (doctoral thesis). University of Adelaide, Adelaide. >> >> Yuan, K. H., & Bentler, P. M. (2000). Three likelihood-based methods for >> mean and covariance structure analysis with nonnormal missing data. In >> M. E. Sobel & M. P. Becker (Eds.), Sociological Methodology (pp. >> 165-200). Washington, D.C: ASA. >> >> In addition to elaborating, better yet, if you have a moment give us >> some syntax for a dataset that you can create where there are >> simultaneous equations with observed variables, an omitted cause, and >> instruments. Let's see how the Hansen-J test (estimated with reg3, with >> 2sls and 3sls) and the normal theory chi-square statistic (estimated >> with sem) behave (with and with robust corrections). >> >> Best, >> J. >> >> __________________________________________ >> >> John Antonakis >> Professor of Organizational Behavior >> Director, Ph.D. Program in Management >> >> Faculty of Business and Economics >> University of Lausanne >> Internef #618 >> CH-1015 Lausanne-Dorigny >> Switzerland >> Tel ++41 (0)21 692-3438 >> Fax ++41 (0)21 692-3305 >> http://www.hec.unil.ch/people/jantonakis >> >> Associate Editor >> The Leadership Quarterly >> __________________________________________ >> >> On 16.04.2013 22:04, Lachenbruch, Peter wrote: >>> >>> I would be rather cautious about relying on tests of variances. These >>> are notoriously non-robust. Unless new theory has shown this not to be the >>> case, i'd not regard this as a major issue. >>> >>> Peter A. Lachenbruch, >>> Professor (retired) >>> ________________________________________ >>> From: owner-statalist@hsphsun2.harvard.edu >>> [owner-statalist@hsphsun2.harvard.edu] on behalf of John Antonakis >>> [John.Antonakis@unil.ch] >>> Sent: Tuesday, April 16, 2013 10:51 AM >>> To: statalist@hsphsun2.harvard.edu >>> Subject: Re: st: Query.. >>> >>> In general I find Acock's books helpful and I have bought two of them. >>> The latest one he has on SEM was gives a very nice overview of the SEM >>> module in Stata. However, it is disappointing on some of the statistical >>> theory, in particular with respect to fact that he gave too much >>> coverage to "approximate" indexes of overidentification, which are not >>> tests, and did not explain enough what the chi-square statistic of >>> overidentification is. >>> >>> The Stata people are usually very good about strictly following >>> statistical theory, as do all econometricians, and do not promote too >>> much these approximate indexes. So, I was a bit annoyed to see how much >>> airtime was given to rule-of-thumb indexes that have no known >>> distributions and are not tests. The only serious test of >>> overidentification, analogous to the Hansen-Sargen statistic is the >>> chi-square test of fit. So, my suggestion to Alan is that he spends some >>> time to cover that in the updated addition and not to suggest that >>> models that fail the chi-square test are "approximately good." >>> >>> For those who do not know what this statistic does, it basically >>> compares the observed variance-covariance (S) matrix to the fitted >>> variance covariance matrix (sigma) to see if the difference (residuals) >>> are simultaneously different from zero. The fitting function that is >>> minimized is: >>> >>> Fml = ln|Sigma| - ln|S| + trace[S.Sigma^-1] - p >>> >>> As Sigma approaches S, the log of the determinant of Sigma less the log >>> of the determinant of S approach zero; as concerns the two last terms, >>> as Sigma approaches S, the inverse of Sigma premultiplied by S makes an >>> identity matrix, whose trace will equal the number of observed variables >>> p (thus, those two terms also approach zero). The chi-square statistic >>> is simply Fml*N, at the relevant DF (which is elements in the >>> variance-covariance matrix less parameters estimated). >>> >>> This chi-square test will not reject a correctly specified model; >>> however, it does not behave as expected at small samples, when data are >>> not multivariate normal, when the model is complex (and the n to >>> parameters estimated ration is low), which is why several corrections >>> have been shown to better approximate the true chi-square distribution >>> (e.g., Swain correction, Yuan-Bentler correction, Bollen-Stine >>> bootstrap). >>> >>> In all, I am thankful to Alan for his nice "how-to" guides which are >>> very helpful to students who do not know Stata need a "gentle >>> introduction"--so I recommend them to my students, that is for sure. >>> But, I would appreciate a bit more beef from him for the SEM book in >>> updated versions. >>> >>> Best, >>> J. >>> >>> __________________________________________ >>> >>> John Antonakis >>> Professor of Organizational Behavior >>> Director, Ph.D. Program in Management >>> >>> Faculty of Business and Economics >>> University of Lausanne >>> Internef #618 >>> CH-1015 Lausanne-Dorigny >>> Switzerland >>> Tel ++41 (0)21 692-3438 >>> Fax ++41 (0)21 692-3305 >>> http://www.hec.unil.ch/people/jantonakis >>> >>> Associate Editor >>> The Leadership Quarterly >>> __________________________________________ >>> >>> On 16.04.2013 17:45, Lachenbruch, Peter wrote: >>> > David - >>> > It would be good for you to specify what you find problematic with >>> Acock's book. I've used it and not had any problems - but maybe i'm >>> just ancient and not seeing issues >>> > >>> > Peter A. Lachenbruch, >>> > Professor (retired) >>> > ________________________________________ >>> > From: owner-statalist@hsphsun2.harvard.edu >>> [owner-statalist@hsphsun2.harvard.edu] on behalf of Hutagalung, Robert >>> [Robert.Hutagalung@med.uni-jena.de] >>> > Sent: Monday, April 15, 2013 2:06 AM >>> > To: statalist@hsphsun2.harvard.edu >>> > Subject: AW: st: Query.. >>> > >>> > Hi David, >>> > Thanks, though I find the book very useful. >>> > Best, Rob >>> > >>> > -----Ursprüngliche Nachricht----- >>> > Von: owner-statalist@hsphsun2.harvard.edu >>> [mailto:owner-statalist@hsphsun2.harvard.edu] Im Auftrag von David >>> Hoaglin >>> > Gesendet: Samstag, 13. April 2013 16:11 >>> > An: statalist@hsphsun2.harvard.edu >>> > Betreff: Re: st: Query.. >>> > >>> > Hi, Rob. >>> > >>> > I am not able to suggest a book on >>> pharmacokinetics/pharmacodynamics, >>> > but I do have a comment on A Gentle Introduction to Stata. As a >>> statistician, I found it helpful in learning to use Stata, but a number >>> of its explanations of statistics are very worrisome. >>> > >>> > David Hoaglin >>> > >>> > On Fri, Apr 12, 2013 at 9:01 AM, Hutagalung, Robert >>> <Robert.Hutagalung@med.uni-jena.de> wrote: >>> >> Hi everyone, I am a new fellow here.. >>> >> I am wondering if somebody could a book (or books) on Stata >>> dealing >>> with pharmacokinetics/pharmacodinamics - both analyses and graphs. >>> >> I already have: A Visual Guide to Stata Graphics, 2' Edition, A >>> Gentle Introduction to Stata, Third Edition, An Introduction to Stata >>> for Health Researchers, Third Edition. >>> >> Thanks in advance, Rob. >>> > * >>> > * For searches and help try: >>> > * http://www.stata.com/help.cgi?search >>> > * http://www.stata.com/support/faqs/resources/statalist-faq/ >>> > * http://www.ats.ucla.edu/stat/stata/ >>> > >>> > Universitätsklinikum Jena - Bachstrasse 18 - D-07743 Jena >>> > Die gesetzlichen Pflichtangaben finden Sie unter >>> http://www.uniklinikum-jena.de/Pflichtangaben.html >>> > >>> > * >>> > * For searches and help try: >>> > * http://www.stata.com/help.cgi?search >>> > * http://www.stata.com/support/faqs/resources/statalist-faq/ >>> > * http://www.ats.ucla.edu/stat/stata/ >>> > * >>> > * For searches and help try: >>> > * http://www.stata.com/help.cgi?search >>> > * http://www.stata.com/support/faqs/resources/statalist-faq/ >>> > * http://www.ats.ucla.edu/stat/stata/ >>> >>> * >>> * For searches and help try: >>> * http://www.stata.com/help.cgi?search >>> * http://www.stata.com/support/faqs/resources/statalist-faq/ >>> * http://www.ats.ucla.edu/stat/stata/ >>> * >>> * For searches and help try: >>> * http://www.stata.com/help.cgi?search >>> * http://www.stata.com/support/faqs/resources/statalist-faq/ >>> * http://www.ats.ucla.edu/stat/stata/ >> >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/faqs/resources/statalist-faq/ >> * http://www.ats.ucla.edu/stat/stata/ >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/faqs/resources/statalist-faq/ >> * http://www.ats.ucla.edu/stat/stata/ > > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/faqs/resources/statalist-faq/ > * http://www.ats.ucla.edu/stat/stata/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/faqs/resources/statalist-faq/ * http://www.ats.ucla.edu/stat/stata/