Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: Query..
From
Nick Cox <[email protected]>
To
"[email protected]" <[email protected]>
Subject
Re: st: Query..
Date
Wed, 17 Apr 2013 11:37:15 +0100
You are simulating well-behaved data -- I see -rnormal()- everywhere
-- but the doubts raised are about how well tests perform in
non-standard situations.
Nick
[email protected]
On 17 April 2013 11:28, John Antonakis <[email protected]> wrote:
> OK....fine with this, but it has no bearing whatsoever on what I said below,
> which was on overidentification and how it is tested in SEM. The chi-square
> statistic of SEM will be very similar to the overidentification statistic
> from a 2sls or 3sls model, and that is the point I was trying to get across.
> So, if economists (and others) trust the Hansen-Sargan overidentification
> statistic, then they should trust and sem chi-square overidentification
> statistic (and not indexes that are not tests).
>
> Run this code to see that we can about the same chi-square value whether
> using 2sls or sem, even though they go about it in very different way (i.e.,
> in 2sls, chi = r-square*N from a regression of the residuals of the y
> equation on the excluded in instruments, whereas the chi-square test from
> SEM uses the discrepancy function I showed below from sigmal and S):
>
> clear
> set seed 123
> set obs 1000
>
> gen x1 = rnormal()
> gen x2 = rnormal()
> gen q = rnormal()
> gen m = x1 + x2 - q + rnormal()
> gen y = m + q + rnormal()
>
> qui: ivregress 2sls y (m = x1 x2)
> qui: estat overid
> scalar chi_sargan = r(sargan)
> scalar p_sargan = r(p_sargan)
>
> qui: sem (y<-m) (m<-x1 x2), cov(e.y*e.m)
> qui: estat gof
> scalar chi_sem = r(chi2_ms)
> scalar p_sem = r(p_ms)
>
> dis "Chi Sargan = "chi_sargan ", p-value = "p_sargan
> dis "Chi SEM = "chi_sem ", p-value = "p_sem
>
> Now, having latent variable in there does not change the basis of how this
> chi-square statistic is calculated in SEM. Also, if we have one of the
> conditions that makes the chi-square misbehave (that I identified below),
> then we can rescale the SEM chi-square using one of the corrective
> procedures so that it approximates the expected chi-square distribution.
>
>
> Best,
> J.
>
>
> __________________________________________
>
> John Antonakis
> Professor of Organizational Behavior
> Director, Ph.D. Program in Management
>
> Faculty of Business and Economics
> University of Lausanne
> Internef #618
> CH-1015 Lausanne-Dorigny
> Switzerland
> Tel ++41 (0)21 692-3438
> Fax ++41 (0)21 692-3305
> http://www.hec.unil.ch/people/jantonakis
>
> Associate Editor
> The Leadership Quarterly
> __________________________________________
>
> On 17.04.2013 01:01, Lachenbruch, Peter wrote:
>>
>> The context i was referring to was an old article by George Box in
>> Biometrika aboutg 1953 in which he commented that testing for
>> heteroskedasticy was like setting to see in a rowboat to see if it was safe
>> for the Queen Mary to sail. Sorry i don't have the quote, and my books are
>> all bundled up due to a flood in my basement
>>
>> Peter A. Lachenbruch,
>> Professor (retired)
>> ________________________________________
>> From: [email protected]
>> [[email protected]] on behalf of John Antonakis
>> [[email protected]]
>> Sent: Tuesday, April 16, 2013 1:47 PM
>> To: [email protected]
>> Subject: Re: st: Query..
>>
>> Hello Peter:
>>
>> Can you please elaborate? The chi-square test of fit--or the likelihood
>> ratio test comparing the saturated to the target model--is pretty
>> robust, though as I indicated, it does not behave as expected at small
>> samples, when data are not multivariate normal, when the model is
>> complex (and the n to parameters estimated ration is low). However, as I
>> mentioned there are remedies to the problem. More specifically see:
>>
>> Bollen, K. A., & Stine, R. A. (1992). Bootstrapping goodness-of-fit
>> measures in structural equation models. Sociological Methods & Research,
>> 21(2), 205-229.
>>
>> Herzog, W., & Boomsma, W. (2009). Small-sample robust estimators of
>> noncentrality-based and incremental model fit. Structural Equation
>> Modeling, 16(1), 1–27.
>>
>> Swain, A. J. (1975). Analysis of parametric structures for variance
>> matrices (doctoral thesis). University of Adelaide, Adelaide.
>>
>> Yuan, K. H., & Bentler, P. M. (2000). Three likelihood-based methods for
>> mean and covariance structure analysis with nonnormal missing data. In
>> M. E. Sobel & M. P. Becker (Eds.), Sociological Methodology (pp.
>> 165-200). Washington, D.C: ASA.
>>
>> In addition to elaborating, better yet, if you have a moment give us
>> some syntax for a dataset that you can create where there are
>> simultaneous equations with observed variables, an omitted cause, and
>> instruments. Let's see how the Hansen-J test (estimated with reg3, with
>> 2sls and 3sls) and the normal theory chi-square statistic (estimated
>> with sem) behave (with and with robust corrections).
>>
>> Best,
>> J.
>>
>> __________________________________________
>>
>> John Antonakis
>> Professor of Organizational Behavior
>> Director, Ph.D. Program in Management
>>
>> Faculty of Business and Economics
>> University of Lausanne
>> Internef #618
>> CH-1015 Lausanne-Dorigny
>> Switzerland
>> Tel ++41 (0)21 692-3438
>> Fax ++41 (0)21 692-3305
>> http://www.hec.unil.ch/people/jantonakis
>>
>> Associate Editor
>> The Leadership Quarterly
>> __________________________________________
>>
>> On 16.04.2013 22:04, Lachenbruch, Peter wrote:
>>>
>>> I would be rather cautious about relying on tests of variances. These
>>> are notoriously non-robust. Unless new theory has shown this not to be the
>>> case, i'd not regard this as a major issue.
>>>
>>> Peter A. Lachenbruch,
>>> Professor (retired)
>>> ________________________________________
>>> From: [email protected]
>>> [[email protected]] on behalf of John Antonakis
>>> [[email protected]]
>>> Sent: Tuesday, April 16, 2013 10:51 AM
>>> To: [email protected]
>>> Subject: Re: st: Query..
>>>
>>> In general I find Acock's books helpful and I have bought two of them.
>>> The latest one he has on SEM was gives a very nice overview of the SEM
>>> module in Stata. However, it is disappointing on some of the statistical
>>> theory, in particular with respect to fact that he gave too much
>>> coverage to "approximate" indexes of overidentification, which are not
>>> tests, and did not explain enough what the chi-square statistic of
>>> overidentification is.
>>>
>>> The Stata people are usually very good about strictly following
>>> statistical theory, as do all econometricians, and do not promote too
>>> much these approximate indexes. So, I was a bit annoyed to see how much
>>> airtime was given to rule-of-thumb indexes that have no known
>>> distributions and are not tests. The only serious test of
>>> overidentification, analogous to the Hansen-Sargen statistic is the
>>> chi-square test of fit. So, my suggestion to Alan is that he spends some
>>> time to cover that in the updated addition and not to suggest that
>>> models that fail the chi-square test are "approximately good."
>>>
>>> For those who do not know what this statistic does, it basically
>>> compares the observed variance-covariance (S) matrix to the fitted
>>> variance covariance matrix (sigma) to see if the difference (residuals)
>>> are simultaneously different from zero. The fitting function that is
>>> minimized is:
>>>
>>> Fml = ln|Sigma| - ln|S| + trace[S.Sigma^-1] - p
>>>
>>> As Sigma approaches S, the log of the determinant of Sigma less the log
>>> of the determinant of S approach zero; as concerns the two last terms,
>>> as Sigma approaches S, the inverse of Sigma premultiplied by S makes an
>>> identity matrix, whose trace will equal the number of observed variables
>>> p (thus, those two terms also approach zero). The chi-square statistic
>>> is simply Fml*N, at the relevant DF (which is elements in the
>>> variance-covariance matrix less parameters estimated).
>>>
>>> This chi-square test will not reject a correctly specified model;
>>> however, it does not behave as expected at small samples, when data are
>>> not multivariate normal, when the model is complex (and the n to
>>> parameters estimated ration is low), which is why several corrections
>>> have been shown to better approximate the true chi-square distribution
>>> (e.g., Swain correction, Yuan-Bentler correction, Bollen-Stine
>>> bootstrap).
>>>
>>> In all, I am thankful to Alan for his nice "how-to" guides which are
>>> very helpful to students who do not know Stata need a "gentle
>>> introduction"--so I recommend them to my students, that is for sure.
>>> But, I would appreciate a bit more beef from him for the SEM book in
>>> updated versions.
>>>
>>> Best,
>>> J.
>>>
>>> __________________________________________
>>>
>>> John Antonakis
>>> Professor of Organizational Behavior
>>> Director, Ph.D. Program in Management
>>>
>>> Faculty of Business and Economics
>>> University of Lausanne
>>> Internef #618
>>> CH-1015 Lausanne-Dorigny
>>> Switzerland
>>> Tel ++41 (0)21 692-3438
>>> Fax ++41 (0)21 692-3305
>>> http://www.hec.unil.ch/people/jantonakis
>>>
>>> Associate Editor
>>> The Leadership Quarterly
>>> __________________________________________
>>>
>>> On 16.04.2013 17:45, Lachenbruch, Peter wrote:
>>> > David -
>>> > It would be good for you to specify what you find problematic with
>>> Acock's book. I've used it and not had any problems - but maybe i'm
>>> just ancient and not seeing issues
>>> >
>>> > Peter A. Lachenbruch,
>>> > Professor (retired)
>>> > ________________________________________
>>> > From: [email protected]
>>> [[email protected]] on behalf of Hutagalung, Robert
>>> [[email protected]]
>>> > Sent: Monday, April 15, 2013 2:06 AM
>>> > To: [email protected]
>>> > Subject: AW: st: Query..
>>> >
>>> > Hi David,
>>> > Thanks, though I find the book very useful.
>>> > Best, Rob
>>> >
>>> > -----Ursprüngliche Nachricht-----
>>> > Von: [email protected]
>>> [mailto:[email protected]] Im Auftrag von David
>>> Hoaglin
>>> > Gesendet: Samstag, 13. April 2013 16:11
>>> > An: [email protected]
>>> > Betreff: Re: st: Query..
>>> >
>>> > Hi, Rob.
>>> >
>>> > I am not able to suggest a book on
>>> pharmacokinetics/pharmacodynamics,
>>> > but I do have a comment on A Gentle Introduction to Stata. As a
>>> statistician, I found it helpful in learning to use Stata, but a number
>>> of its explanations of statistics are very worrisome.
>>> >
>>> > David Hoaglin
>>> >
>>> > On Fri, Apr 12, 2013 at 9:01 AM, Hutagalung, Robert
>>> <[email protected]> wrote:
>>> >> Hi everyone, I am a new fellow here..
>>> >> I am wondering if somebody could a book (or books) on Stata
>>> dealing
>>> with pharmacokinetics/pharmacodinamics - both analyses and graphs.
>>> >> I already have: A Visual Guide to Stata Graphics, 2' Edition, A
>>> Gentle Introduction to Stata, Third Edition, An Introduction to Stata
>>> for Health Researchers, Third Edition.
>>> >> Thanks in advance, Rob.
>>> > *
>>> > * For searches and help try:
>>> > * http://www.stata.com/help.cgi?search
>>> > * http://www.stata.com/support/faqs/resources/statalist-faq/
>>> > * http://www.ats.ucla.edu/stat/stata/
>>> >
>>> > Universitätsklinikum Jena - Bachstrasse 18 - D-07743 Jena
>>> > Die gesetzlichen Pflichtangaben finden Sie unter
>>> http://www.uniklinikum-jena.de/Pflichtangaben.html
>>> >
>>> > *
>>> > * For searches and help try:
>>> > * http://www.stata.com/help.cgi?search
>>> > * http://www.stata.com/support/faqs/resources/statalist-faq/
>>> > * http://www.ats.ucla.edu/stat/stata/
>>> > *
>>> > * For searches and help try:
>>> > * http://www.stata.com/help.cgi?search
>>> > * http://www.stata.com/support/faqs/resources/statalist-faq/
>>> > * http://www.ats.ucla.edu/stat/stata/
>>>
>>> *
>>> * For searches and help try:
>>> * http://www.stata.com/help.cgi?search
>>> * http://www.stata.com/support/faqs/resources/statalist-faq/
>>> * http://www.ats.ucla.edu/stat/stata/
>>> *
>>> * For searches and help try:
>>> * http://www.stata.com/help.cgi?search
>>> * http://www.stata.com/support/faqs/resources/statalist-faq/
>>> * http://www.ats.ucla.edu/stat/stata/
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/faqs/resources/statalist-faq/
>> * http://www.ats.ucla.edu/stat/stata/
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/faqs/resources/statalist-faq/
>> * http://www.ats.ucla.edu/stat/stata/
>
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/faqs/resources/statalist-faq/
> * http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/faqs/resources/statalist-faq/
* http://www.ats.ucla.edu/stat/stata/