Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: AW: gllamm (poisson) execution time


From   Jeph Herrin <junk@spandrel.net>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: AW: gllamm (poisson) execution time
Date   Thu, 25 Jun 2009 10:43:16 -0400

If you have a single random effect, you may find -xtpoisson-
is even faster than -xtmepoisson-.

hth,
Jeph


Keith Dear (home) wrote:
Ummm ... no (well, NOW I have).
Except on the uni supercomputer, we only have Stata9, hence ignorance. Time to upgrade!
Many thanks Martin.
Keith

ps
http://www.stata.com/help.cgi?xtmepoisson
http://stata.com/stata10/mixedmodels.html



Martin Weiss wrote:
<>
Have you looked into -xtmepoisson-?




HTH
Martin

-----Ursprüngliche Nachricht-----
Von: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] Im Auftrag von Keith Dear
(work)
Gesendet: Mittwoch, 24. Juni 2009 08:01
An: statalist@hsphsun2.harvard.edu
Cc: Ainslie Butler
Betreff: st: gllamm (poisson) execution time

We are trying to model daily mortality by poisson regression, over 17 years, by postcode, with postcode as a single random intercept term. In Stata10/MP4 on a linux cluster our models each take 7 or 8 hours to fit, which is too long to be feasible for exploratory analyses.

The full dataset has >14 million rows of data: a row for every day for 1991-2007 for every postcode in Australia (~2200 postcodes), but to get things moving we are starting with smaller geographical regions of only 100 or 200 postcodes. Thus N=17*365*(100 or 200), about a half or one million. Also we are starting with failrly simple models, p=17 fixed-effect parameters just for trend and annual cycles. The models converge ok, eventually, in only a few iterations and with typical condition number about 2.

I found this in the list archives (from Sophia Rabe-Hesketh in 2003):
==> biggest gain is to reduce M, followed by n, p and N
Here we have M=1, n=5 (down from the default of 8), p=17, but N=6E5 or more. There does not seem to be much prospect of reducing any of those, indeed we will need to substantially increase p (for more interesting models) and N (to cover all of Australia at once).

Is there hope? Are there alternatives to gllamm for this? Or are we overlooking something basic here?
Keith


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index