Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: "Can Your Results be Replicated?" (Stata error?)
From
Stas Kolenikov <[email protected]>
To
"[email protected]" <[email protected]>
Subject
Re: st: "Can Your Results be Replicated?" (Stata error?)
Date
Fri, 13 Sep 2013 09:54:55 -0500
Anders, thanks for the rigorous follow up.
-- Stas Kolenikov, PhD, PStat (ASA, SSC)
-- Senior Survey Statistician, Abt SRBI
-- Opinions stated in this email are mine only, and do not reflect the
position of my employer
-- http://stas.kolenikov.name
On Fri, Sep 13, 2013 at 9:51 AM, Anders Alexandersson
<[email protected]> wrote:
> Joerg is correct, and I agree with Stas. I contacted the author Mark
> Bell who replied:
>
> "We never claim the error was in the firth logit command. The problem
> was that Rauchhaus was using a logit GEE as a way to deal with
> separation (xtgee). GEE is not a way to deal with separation, and the
> model should have been unidentified, but stata nonetheless returned a
> coefficient on the variable causing separation. This is the error we
> were referring to."
>
> Anders
>
> On Fri, Sep 13, 2013 at 10:15 AM, Stas Kolenikov <[email protected]> wrote:
>> Maybe it's time Stata Corp picks up -firthlogit-, solidifies it and
>> makes it an official command.
>>
>> -- Stas Kolenikov, PhD, PStat (ASA, SSC)
>> -- Senior Survey Statistician, Abt SRBI
>> -- Opinions stated in this email are mine only, and do not reflect the
>> position of my employer
>> -- http://stas.kolenikov.name
>>
>>
>>
>> On Fri, Sep 13, 2013 at 9:01 AM, Joerg Luedicke
>> <[email protected]> wrote:
>>> After having a quick glance at their paper
>>> (http://jcr.sagepub.com/content/early/2013/08/19/0022002713499718.abstract?papetoc)
>>> it seems that they are talking about a problem with Stata's -xtgee-
>>> command which, in the case of separation in a logit model, provides
>>> nonsense results as opposed to omitting predictors or the like. Below
>>> is a toy example showing what seems to be the problem. However,
>>> finding an effect of something like "x is 3 million times less likely
>>> than y" and not getting suspicious rather looks like sloppy research
>>> to me in the first place.
>>>
>>> Joerg
>>>
>>>
>>> *------------------------
>>> clear
>>> set obs 100
>>> set seed 123
>>>
>>> gen id = _n
>>> gen ui = rnormal(0,0.5)
>>>
>>> expand 10
>>> bys id : gen year = _n
>>> gen x = cond(mod(_n-1, 3) == 1, 1, cond(mod(_n-1, 3) == 0, 2, 3))
>>> tab x, gen(x_)
>>>
>>> gen xb = 1 / (1 + exp(-(0.3*x_2 + 0.3*x_3 + ui)))
>>> gen y = rbinomial(1,xb)
>>> replace y = 0 if x_1 == 1
>>> tab y x
>>>
>>> xtset id year
>>> xtgee y i.x, fam(binomial) link(logit)
>>> melogit y i.x || id:
>>> logit y i.x
>>> *------------------------
>>>
>>> On Fri, Sep 13, 2013 at 10:41 AM, Richard Williams
>>> <[email protected]> wrote:
>>>> At 08:14 AM 9/13/2013, Anders Alexandersson wrote:
>>>>>
>>>>> I just made this reply on the blog:
>>>>>
>>>>> "Where is the error in Stata? The author’s so called “Do-File for
>>>>> Analyses.txt” is actually not a Stata do file but it does refer to
>>>>> Stata’s user-written command -firthlogit- from SSC. Please provide a
>>>>> reproducible do-file in Stata.The claim that results and conclusions
>>>>> were due to an error in Stata is not supported."
>>>>> See
>>>>> http://politicalsciencereplication.wordpress.com/2013/09/11/guest-blog-how-to-persuade-journals-to-accept-your-replication-paper/comment-page-1/#comment-653
>>>>
>>>>
>>>> Even if -firthlogit- did get it wrong, it is a bit of a stretch to imply
>>>> that Stata has some terrible flaw. Stata Corp can hardly be held responsible
>>>> for flaws in programs it did not write.
>>>>
>>>> When I first sent a program to SSC, I thought there might be some sort of
>>>> exhaustive review process before it was released to the public. I got the
>>>> feeling that wasn't the case when I got a message less than an hour later
>>>> saying the program had been posted. Most user-written routines are fine but
>>>> people should realize they haven't undergone the kind of testing that
>>>> official programs have. And even in this case, we don't have any proof yet
>>>> that firthlogit did get it wrong.
>>>>
>>>>
>>>>> Anders Alexandersson
>>>>> [email protected]
>>>>>
>>>>> On Fri, Sep 13, 2013 at 8:10 AM, Philip Jones
>>>>> <[email protected]> wrote:
>>>>> > Hi all,
>>>>> >
>>>>> > I found a link on my Twitter feed this AM, purporting to show how
>>>>> > Stata "made a mistake" that R did not make:
>>>>> >
>>>>> > http://www.r-bloggers.com/can-your-results-be-replicated/
>>>>> >
>>>>> > which actually points to:
>>>>> >
>>>>> >
>>>>> > http://politicalsciencereplication.wordpress.com/2013/09/11/guest-blog-how-to-persuade-journals-to-accept-your-replication-paper/
>>>>> >
>>>>> > I realize that "r-bloggers" is likely not the most bias-free site when
>>>>> > it comes to reviewing/rating stats packages, but has anyone got an
>>>>> > idea as to what is actually going on here? Is Stata really at fault?
>>>>> >
>>>>> > Regards,
>>>>> >
>>>>> > Phil
>>>>> > @pmgjones on Twitter
>>>>> > *
>>>>> > * For searches and help try:
>>>>> > * http://www.stata.com/help.cgi?search
>>>>> > * http://www.stata.com/support/faqs/resources/statalist-faq/
>>>>> > * http://www.ats.ucla.edu/stat/stata/
>>>>>
>>>>> *
>>>>> * For searches and help try:
>>>>> * http://www.stata.com/help.cgi?search
>>>>> * http://www.stata.com/support/faqs/resources/statalist-faq/
>>>>> * http://www.ats.ucla.edu/stat/stata/
>>>>
>>>>
>>>> -------------------------------------------
>>>> Richard Williams, Notre Dame Dept of Sociology
>>>> OFFICE: (574)631-6668, (574)631-6463
>>>> HOME: (574)289-5227
>>>> EMAIL: [email protected]
>>>> WWW: http://www.nd.edu/~rwilliam
>>>>
>>>>
>>>>
>>>> *
>>>> * For searches and help try:
>>>> * http://www.stata.com/help.cgi?search
>>>> * http://www.stata.com/support/faqs/resources/statalist-faq/
>>>> * http://www.ats.ucla.edu/stat/stata/
>>>
>>> *
>>> * For searches and help try:
>>> * http://www.stata.com/help.cgi?search
>>> * http://www.stata.com/support/faqs/resources/statalist-faq/
>>> * http://www.ats.ucla.edu/stat/stata/
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/faqs/resources/statalist-faq/
>> * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/faqs/resources/statalist-faq/
> * http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/faqs/resources/statalist-faq/
* http://www.ats.ucla.edu/stat/stata/