[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
"Keith Dear (home)" <keith.dear@anu.edu.au> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: AW: gllamm (poisson) execution time |

Date |
Sat, 27 Jun 2009 10:32:33 +1000 |

Yes, with -adapt- gllamm returns a variance estimate of .09534258, which now agrees to 2sigfig with the xtpoisson estimate. So adaptive quadrature would seem (N=1) to be a good thing, though it did increase execution time from 5 to 15 sec. Regardless, xtpoisson rules for our simple models (but in Stata11, naturally :) ) Thanks again to all who responded, Keith Roberto G. Gutierrez wrote:

Keith,You're suspicions are correct. Because -xtpoisson- uses morespecicalizedcode it is more accurate than -gllamm- for this particular model and on this particular data. Another recommendation: Try using option -adapt- in -gllamm- to get adaptive quadrature rather than regular quadrature. That should helpbridge the gap between -xtpoisson- and -gllamm-; -xtpoisson- usesadaptive quadrature (although not precisely the same species of itthan -gllamm-).--Bobby On Fri, 26 Jun 2009, Keith Dear (home) wrote:Thanks to all who have pointed this out, including Roberto G.Gutierrez who was first, but off list.You are not wrong about the speed: 8 hours in gllamm, 4 minutes inxtpoisson!! (in MP4)But it's disturbing how different the results can be. In this example(suggested by RGG), the variance estimates don't agree to even onefigure on what I think are equivalent models, or aren't they?webuse ships, clear gen logserv=ln(service) glo X op_75_79 co_65_69 co_70_74 co_75_79 xtset shipxtpoisson accident $X, offset(logserv) normal //takes 0.14 seconds on my pcgllamm accident $X, fam(poisson) offset(logserv) i(ship) //takes 5.36 seconds on my pc~~~~~~~~~ xtpoisson results ~~~~~~~~~------------------------------------------------------------------------------accident | Coef. Std. Err. z P>|z| [95% Conf.Interval]-------------+----------------------------------------------------------------op_75_79 | .3830105 .118253 3.24 0.001 .1512389.6147821co_65_69 | .7093762 .149593 4.74 0.000 .41617941.002573co_70_74 | .8576789 .1693625 5.06 0.000 .52573461.189623co_75_79 | .4992132 .2317164 2.15 0.031 .0450574.953369_cons | -6.640989 .2067838 -32.12 0.000 -7.046278-6.2357logserv | (offset)-------------+----------------------------------------------------------------/lnsig2u | -2.352979 .8583287 -2.74 0.006 -4.035272-.6706858-------------+----------------------------------------------------------------sigma_u | .3083593 .1323368 .1329694.7150928------------------------------------------------------------------------------Likelihood-ratio test of sigma_u=0: chibar2(01) = 10.67Pr>=chibar2 = 0.001~~~~~~~~~ gllamm results ~~~~~~~~~------------------------------------------------------------------------------accident | Coef. Std. Err. z P>|z| [95% Conf.Interval]-------------+----------------------------------------------------------------op_75_79 | .3849786 .1182184 3.26 0.001 .1532747.6166824co_65_69 | .7058854 .1495483 4.72 0.000 .412776.9989947co_70_74 | .847284 .1692169 5.01 0.000 .51562491.178943co_75_79 | .4940048 .2301141 2.15 0.032 .0429894.9450201_cons | -6.724426 .140161 -47.98 0.000 -6.999137-6.449716logserv | (offset)------------------------------------------------------------------------------Variances and covariances of random effects-----------------------------------------------------------------------------***level 2 (ship) var(1): .17662891 (.09378635)-----------------------------------------------------------------------------My (quite likely wrong) understanding of these results is thatexp(-2.352979)= 0.095085 and .17662891 are estimates of the samevariance parameter, which is a bit worrying. I take it the value(.09378635) is the SE of the variance estimate, and it's just acoincidence that it happens to be close to the xtpoisson varianceestimate.Increasing the nip() parameter of gllamm from the default 8 to 19changes the 0.1766.. value to 0.3529.., which suggests to me that thextpoisson result is perhaps more reliable (it also doubles theexecution time to 10.31 sec). Can someone more expert confirm and/orexplain? We know that precise is not the same as accurate, so perhapsinvariant is also not to the point.Thanks Keith Jeph Herrin wrote:If you have a single random effect, you may find -xtpoisson- is even faster than -xtmepoisson-. hth, Jeph Keith Dear (home) wrote:Ummm ... no (well, NOW I have).Except on the uni supercomputer, we only have Stata9, henceignorance. Time to upgrade!Many thanks Martin. Keith ps http://www.stata.com/help.cgi?xtmepoisson http://stata.com/stata10/mixedmodels.html Martin Weiss wrote:<> Have you looked into -xtmepoisson-? HTH Martin -----Ursprüngliche Nachricht----- Von: owner-statalist@hsphsun2.harvard.edu[mailto:owner-statalist@hsphsun2.harvard.edu] Im Auftrag von KeithDear(work) Gesendet: Mittwoch, 24. Juni 2009 08:01 An: statalist@hsphsun2.harvard.edu Cc: Ainslie Butler Betreff: st: gllamm (poisson) execution timeWe are trying to model daily mortality by poisson regression, over17 years, by postcode, with postcode as a single random interceptterm.In Stata10/MP4 on a linux cluster our models each take 7 or 8hours to fit, which is too long to be feasible for exploratoryanalyses.The full dataset has >14 million rows of data: a row for every dayfor 1991-2007 for every postcode in Australia (~2200 postcodes),but to get things moving we are starting with smaller geographicalregions of only 100 or 200 postcodes. Thus N=17*365*(100 or 200),about a half or one million. Also we are starting with failrlysimple models, p=17 fixed-effect parameters just for trend andannual cycles. The models converge ok, eventually, in only a fewiterations and with typical condition number about 2.I found this in the list archives (from Sophia Rabe-Hesketh in 2003): ==> biggest gain is to reduce M, followed by n, p and NHere we have M=1, n=5 (down from the default of 8), p=17, butN=6E5 or more. There does not seem to be much prospect of reducingany of those, indeed we will need to substantially increase p (formore interesting models) and N (to cover all of Australia at once).Is there hope? Are there alternatives to gllamm for this? Or arewe overlooking something basic here?Keith* * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/-- Dr Keith Dear Senior Fellow National Centre for Epidemiology and Population Health ANU College of Medicine, Biology and Environment Building 62, cnr Mills and Eggleston Roads Australian National University Canberra ACT 0200 Australia T: 02 6125 4865 F: 02 6125 0740 M: 0424 450 396 W: nceph.anu.edu.au/Staff_Students/staff_pages/dear.php CRICOS provider #00120C http://canberragliding.org/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

-- Dr Keith Dear Senior Fellow National Centre for Epidemiology and Population Health ANU College of Medicine, Biology and Environment Building 62, cnr Mills and Eggleston Roads Australian National University Canberra ACT 0200 Australia T: 02 6125 4865 F: 02 6125 0740 M: 0424 450 396 W: nceph.anu.edu.au/Staff_Students/staff_pages/dear.php CRICOS provider #00120C http://canberragliding.org/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**References**:**st: Interpreting Poisson output***From:*"Data Analytics Corp." <dataanalytics@earthlink.net>

**st: RE: Interpreting Poisson output***From:*"Kieran McCaul" <Kieran.McCaul@uwa.edu.au>

**st: gllamm (poisson) execution time***From:*"Keith Dear (work)" <keith.dear@anu.edu.au>

**Re: st: AW: gllamm (poisson) execution time***From:*"Keith Dear (home)" <keith.dear@anu.edu.au>

**Re: st: AW: gllamm (poisson) execution time***From:*Jeph Herrin <junk@spandrel.net>

**Re: st: AW: gllamm (poisson) execution time***From:*"Keith Dear (home)" <keith.dear@anu.edu.au>

- Prev by Date:
**st: stata2mplus convert mac file name to PC file name** - Next by Date:
**st: tsset with xtivreg2** - Previous by thread:
**Re: st: AW: gllamm (poisson) execution time** - Next by thread:
**st: Fixed Effect Negative Binomial with all zero outcome** - Index(es):

© Copyright 1996–2017 StataCorp LLC | Terms of use | Privacy | Contact us | What's new | Site index |