Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: xtlogit is slow


From   David Airey <david.airey@Vanderbilt.Edu>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: xtlogit is slow
Date   Tue, 8 Jul 2008 11:18:56 -0500

.

It takes 54 seconds on my laptop.

-Dave

. do "/var/folders/AJ/AJjttUSSFdSi1HiInJZyY++++TI/-Tmp-//SD00651.000000"

. clear *

. set more off

. set memory 100M

Current memory allocation

current memory usage
settable value description (1M = 1024k)
--------------------------------------------------------------------
set maxvar 5000 max. variables allowed 2.067M
set memory 100M max. data space 100.000M
set matsize 400 max. RHS vars in models 1.254M
-----------
103.321M

. set seed `=date("2008-07-08", "YMD")'

. set obs 300000
obs was 0, now 300000

. *
. generate byte depvar = uniform() > 0.5

. generate byte catvar1 = floor(4 * uniform())

. forvalue i = 1/12 {
2. generate float other_var`i' = uniform()
3. }

. generate byte rand = floor(9 * uniform())

. generate int panelvar = floor(3000 * uniform())

. timer clear 1

. timer on 1

. xi: xtlogit depvar i.catvar1 other_var* if rand <7, i(panelvar) pa
i.catvar1 _Icatvar1_0-3 (naturally coded; _Icatvar1_0 omitted)

Iteration 1: tolerance = 2.886e-06
Iteration 2: tolerance = 2.193e-10

GEE population-averaged model Number of obs = 233408
Group variable: panelvar Number of groups = 3000
Link: logit Obs per group: min = 47
Family: binomial avg = 77.8
Correlation: exchangeable max = 106
Wald chi2(15) = 18.77
Scale parameter: 1 Prob > chi2 = 0.2245

------------------------------------------------------------------------------
depvar | Coef. Std. Err. z P>|z| [95% Conf. Interval]
------------- +----------------------------------------------------------------
_Icatvar1_1 | .0077015 .0116875 0.66 0.510 -. 0152056 .0306086
_Icatvar1_2 | .0213159 .0117154 1.82 0.069 -. 0016459 .0442778
_Icatvar1_3 | -.0017835 .0117167 -0.15 0.879 -. 0247479 .0211809
other_var1 | .0003397 .0143455 0.02 0.981 -. 0277769 .0284563
other_var2 | -.0074464 .0143168 -0.52 0.603 -. 0355067 .0206139
other_var3 | -.0054181 .0143215 -0.38 0.705 -. 0334876 .0226514
other_var4 | .0065459 .0143327 0.46 0.648 -. 0215456 .0346375
other_var5 | .0086643 .0143549 0.60 0.546 -. 0194707 .0367994
other_var6 | .024404 .0143424 1.70 0.089 -. 0037066 .0525146
other_var7 | .0125822 .0143138 0.88 0.379 -. 0154724 .0406369
other_var8 | -.0029192 .0143368 -0.20 0.839 -. 0310189 .0251804
other_var9 | -.00545 .0143595 -0.38 0.704 -. 0335942 .0226941
other_var10 | .0050038 .0143494 0.35 0.727 -. 0231206 .0331282
other_var11 | -.0380009 .0143391 -2.65 0.008 -.066105 -.0098967
other_var12 | .0203525 .0143351 1.42 0.156 -. 0077437 .0484488
_cons | -.0186582 .0262054 -0.71 0.476 -. 0700199 .0327035
------------------------------------------------------------------------------

. timer off 1

. timer list
1: 54.86 / 1 = 54.8610

. exit

end of do-file



On Jul 8, 2008, at 7:56 AM, Joseph Coveney wrote:


Mitchell F. Berman wrote:

Good thought on the xtlogit issue, but no, the if clause is not the
problem. Takes closer to an hour, I think, for the entire data set, but
I've only done this once. I can tell that it is running slowly, because
each iteration takes many minutes.
I sent this issue in to Stata Tech Support. I'm hoping they investigate it.

--------------------------------------------------------------------------------

I'm curious to see what they discover, because the do file below, which has
15 variables, 300,000 observations, and the command line that you showed
earlier in the thread takes only 2.5 minutes (149.9250 seconds) on my
boat-anchor-disguised-as-a-PC. There must be something very peculiar about
your particular dataset or machine set up.

Joseph Coveney

clear *
set more off
set memory 100M
set seed `=date("2008-07-08", "YMD")'
set obs 300000
*
generate byte depvar = uniform() > 0.5
generate byte catvar1 = floor(4 * uniform())
forvalue i = 1/12 {
generate float other_var`i' = uniform()
}
generate byte rand = floor(9 * uniform())
generate int panelvar = floor(3000 * uniform())
timer clear 1
timer on 1
xi: xtlogit depvar i.catvar1 other_var* if rand <7, i(panelvar) pa
timer off 1
timer list
exit


*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index