[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: AW: gllamm (poisson) execution time

From   Jeph Herrin <>
Subject   Re: st: AW: gllamm (poisson) execution time
Date   Thu, 25 Jun 2009 10:43:16 -0400

If you have a single random effect, you may find -xtpoisson-
is even faster than -xtmepoisson-.


Keith Dear (home) wrote:
Ummm ... no (well, NOW I have).
Except on the uni supercomputer, we only have Stata9, hence ignorance. Time to upgrade!
Many thanks Martin.


Martin Weiss wrote:
Have you looked into -xtmepoisson-?


-----Ursprüngliche Nachricht-----
[] Im Auftrag von Keith Dear
Gesendet: Mittwoch, 24. Juni 2009 08:01
Cc: Ainslie Butler
Betreff: st: gllamm (poisson) execution time

We are trying to model daily mortality by poisson regression, over 17 years, by postcode, with postcode as a single random intercept term. In Stata10/MP4 on a linux cluster our models each take 7 or 8 hours to fit, which is too long to be feasible for exploratory analyses.

The full dataset has >14 million rows of data: a row for every day for 1991-2007 for every postcode in Australia (~2200 postcodes), but to get things moving we are starting with smaller geographical regions of only 100 or 200 postcodes. Thus N=17*365*(100 or 200), about a half or one million. Also we are starting with failrly simple models, p=17 fixed-effect parameters just for trend and annual cycles. The models converge ok, eventually, in only a few iterations and with typical condition number about 2.

I found this in the list archives (from Sophia Rabe-Hesketh in 2003):
==> biggest gain is to reduce M, followed by n, p and N
Here we have M=1, n=5 (down from the default of 8), p=17, but N=6E5 or more. There does not seem to be much prospect of reducing any of those, indeed we will need to substantially increase p (for more interesting models) and N (to cover all of Australia at once).

Is there hope? Are there alternatives to gllamm for this? Or are we overlooking something basic here?

*   For searches and help try:

© Copyright 1996–2022 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index