Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: xtlogit is slow


From   "david.airey@Vanderbilt.Edu" <david.airey@Vanderbilt.Edu>
To   "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Subject   Re: st: xtlogit is slow
Date   Tue, 8 Jul 2008 09:22:50 -0700

.

It takes 54 seconds on my laptop.

-Dave

. do "/var/folders/AJ/AJjttUSSFdSi1HiInJZyY++++TI/-Tmp-//SD00651.000000"

. clear *

. set more off

. set memory 100M

Current memory allocation

                     current                                 memory
usage
     settable          value     description                 (1M =
1024k)

--------------------------------------------------------------------
     set maxvar         5000     max. variables allowed           2.067M
     set memory          100M    max. data space                100.000M
     set matsize         400     max. RHS vars in models          1.254M
                                                             -----------
                                                                103.321M

. set seed `=date("2008-07-08", "YMD")'

. set obs 300000
obs was 0, now 300000

. *
. generate byte depvar = uniform() > 0.5

. generate byte catvar1 = floor(4 * uniform())

. forvalue i = 1/12 {
   2. generate float other_var`i' = uniform()
   3. }

. generate byte rand = floor(9 * uniform())

. generate int panelvar = floor(3000 * uniform())

. timer clear 1

. timer on 1

. xi: xtlogit depvar i.catvar1 other_var* if rand <7, i(panelvar) pa
i.catvar1         _Icatvar1_0-3       (naturally coded; _Icatvar1_0
omitted)

Iteration 1: tolerance = 2.886e-06
Iteration 2: tolerance = 2.193e-10

GEE population-averaged model                   Number of obs
=    233408
Group variable:                   panelvar      Number of groups
=      3000
Link:                                logit      Obs per group: min
=        47
Family:                           binomial                     avg
=      77.8
Correlation:                  exchangeable                     max
=       106
                                                 Wald chi2(15)
=     18.77
Scale parameter:                         1      Prob > chi2
=    0.2245

------------------------------------------------------------------------------
       depvar |      Coef.   Std. Err.      z    P>|z|     [95% Conf.
Interval]
-------------
+----------------------------------------------------------------
  _Icatvar1_1 |   .0077015   .0116875     0.66   0.510    -.
0152056    .0306086
  _Icatvar1_2 |   .0213159   .0117154     1.82   0.069    -.
0016459    .0442778
  _Icatvar1_3 |  -.0017835   .0117167    -0.15   0.879    -.
0247479    .0211809
   other_var1 |   .0003397   .0143455     0.02   0.981    -.
0277769    .0284563
   other_var2 |  -.0074464   .0143168    -0.52   0.603    -.
0355067    .0206139
   other_var3 |  -.0054181   .0143215    -0.38   0.705    -.
0334876    .0226514
   other_var4 |   .0065459   .0143327     0.46   0.648    -.
0215456    .0346375
   other_var5 |   .0086643   .0143549     0.60   0.546    -.
0194707    .0367994
   other_var6 |    .024404   .0143424     1.70   0.089    -.
0037066    .0525146
   other_var7 |   .0125822   .0143138     0.88   0.379    -.
0154724    .0406369
   other_var8 |  -.0029192   .0143368    -0.20   0.839    -.
0310189    .0251804
   other_var9 |    -.00545   .0143595    -0.38   0.704    -.
0335942    .0226941
  other_var10 |   .0050038   .0143494     0.35   0.727    -.
0231206    .0331282
  other_var11 |  -.0380009   .0143391    -2.65   0.008     -.066105
-.0098967
  other_var12 |   .0203525   .0143351     1.42   0.156    -.
0077437    .0484488
        _cons |  -.0186582   .0262054    -0.71   0.476    -.
0700199    .0327035
------------------------------------------------------------------------------

. timer off 1

. timer list
    1:     54.86 /        1 =      54.8610

. exit

end of do-file



On Jul 8, 2008, at 7:56 AM, Joseph Coveney wrote:

> Mitchell F. Berman wrote:
>
> Good thought on the xtlogit issue, but no, the if clause is not the
> problem.  Takes closer to an hour, I think, for the entire data set,
> but
> I've only done this once.  I can tell that it is running slowly,
> because
> each iteration takes many minutes.
> I sent this issue in to Stata Tech Support.  I'm hoping they
> investigate it.
>
> --------------------------------------------------------------------------------
>
> I'm curious to see what they discover, because the do file below,
> which has
> 15 variables, 300,000 observations, and the command line that you
> showed
> earlier in the thread takes only 2.5 minutes (149.9250 seconds) on my
> boat-anchor-disguised-as-a-PC.  There must be something very
> peculiar about
> your particular dataset or machine set up.
>
> Joseph Coveney
>
> clear *
> set more off
> set memory 100M
> set seed `=date("2008-07-08", "YMD")'
> set obs 300000
> *
> generate byte depvar = uniform() > 0.5
> generate byte catvar1 = floor(4 * uniform())
> forvalue i = 1/12 {
> generate float other_var`i' = uniform()
> }
> generate byte rand = floor(9 * uniform())
> generate int panelvar = floor(3000 * uniform())
> timer clear 1
> timer on 1
> xi: xtlogit depvar i.catvar1 other_var* if rand <7, i(panelvar) pa
> timer off 1
> timer list
> exit
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

Privileged, confidential or patient identifiable information may be contained in this message. This information is meant only for the use of the intended recipients. If you are not the intended recipient, or if the message has been addressed to you in error, do not read, disclose, reproduce, distribute, disseminate or otherwise use this transmission. Instead, please notify the sender by reply e-mail, and then destroy all copies of the message and any attachments.

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index