Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: too good to be true : lr test in mlogit?


From   John Litfiba <cariboupad@gmx.fr>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: too good to be true : lr test in mlogit?
Date   Sat, 14 May 2011 11:31:58 +0200

Dear Marten and Joerg,

Thank you for this idea : it can be a interesting solution indeed!
Maybe I shall add 2 remarks here :

1) The log likelihood doesnt converge when I try to fit a random or
fixed effect with xtlogit on my entire dataset..
I have to chose a very "small" (well, compared to the total size of
the sample) of about 10000 observations in order to see the results...
otherwise I get an error message after 3 or 4 iterations

2) The idea of running lets say M regressions over randomly chose
samples could be a solution, but it is statistically valid ? I mean if
I obtain the distribution of the parameters across my M simulation can
I infer something on the parameters of the simulation that should have
been done on the entire dataset ?

Have a good day
Best Regards,

On 13 May 2011 15:11, Joerg Luedicke <joerg.luedicke@gmail.com> wrote:
> On Fri, May 13, 2011 at 6:56 AM, Maarten Buis <maartenlbuis@gmail.com> wrote:
>> On Fri, May 13, 2011 at 11:21 AM, John Litfiba wrote:
>>> But after various tentatives I think that it is not computationally
>>> feasible (at least on my PC) to fit a random or fixed effect logit on
>>> such a large dataset
>>
>> I would not give up on that yet. I would select my model on a sample
>> of your data (I would sample higher level units, rather than
>> individual observations),
>
> This is what I would do, too. In addition, I would be interested in
> the variation of results from different samples. I would recommend
> setting up a program that is drawing a random sample, say 1%, running
> the model, and saving the results. Then let that program run a 1000 or
> whatever times. (How this can be done is described here:
> http://www.stata.com/statalist/archive/2011-04/msg01220.html) Small
> variation in results could then be indicative of the fact that it is
> pointless anyway to have a sample size of > 2mio. In that case, i.e.
> low variation of results, you can also compare your xtlogit model with
> the model you have now etc.  Large variation could be an indication of
> the fact that a very large sample size may be appropriate. You could
> also play with different sample sizes (say, 1%, 3%, 5%) and see how
> the variation of results changes across sample size. At the very
> least, this will provide useful information about your sample and
> might be helpful with respect to the specification of your model.
>
> J.
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index