Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: RE: xtnbreg - same results after convergence at 9,000 iterations or limiting to 100 iterations


From   "William Buchanan" <william@williambuchanan.net>
To   <statalist@hsphsun2.harvard.edu>
Subject   st: RE: xtnbreg - same results after convergence at 9,000 iterations or limiting to 100 iterations
Date   Tue, 25 Jun 2013 10:48:16 -0700

Check out -help maximize-.  It's possible that the model didn't truly
converge, but just reached the default maximum number of iterations.  In
either case, if you see the not concave message repeatedly like that it is
generally a good idea to look at simplifying your model and trying to
identifying which variable is causing the model to fail to converge in a
reasonable amount of time.  

HTH,
Billy

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of nick klein
Sent: Wednesday, June 05, 2013 4:10 PM
To: statalist@hsphsun2.harvard.edu
Subject: st: xtnbreg - same results after convergence at 9,000 iterations or
limiting to 100 iterations

Hi-

I have a question about -xtnbreg-

I'm curious if anyone has any thoughts on I am getting the same results are
when I limit the number of iterations to 100 and when I let the model run
until it converges (sometimes after 10,000 iterations).

I have a large panel dataset and am running a series of models. When I let
the models run overnight, the models converge after somewhere between 50 and
15,000 iterations depending on the model. Looking at the log, I can see that
the models says "(not concave)" through almost all the iterations but do
eventually converge. And the model coefficients seem to make sense.

However, yesterday I was testing out some my code and ran a model where I
limited the number of iterations to 100 (using the iterate(#) option). The
results for the model were exactly the same in both cases
(100 or >10,000 iterations). I can tell that in the steps right before
fitting the final model, the log likelihood for each iteration become very
similar (I assume if I did a trace, I'd see that they are exactly the same)
- i'm just not sure why this might be or what it means for my analysis.

Anyone have thoughts on why this might be?

Below are a few of the relevant commands and then part of each log file -
showing the beginning and final iterations for the xtnbreg model (first
without limiting the number of iterations).

-Thanks
Nick


--------------

. xtset
       panel variable:  FIPSblock (strongly balanced)
        time variable:  Year, 1991 to 2008
                delta:  1 unit

. xtsum Firm_Births

Variable         |      Mean   Std. Dev.       Min        Max |
Observations
-----------------+--------------------------------------------+---------
-----------------+--------------------------------------------+-------
Firm_B~s overall |  .3413659   2.135944          0        141 |     N =
504072
         between |             1.808005          0   69.05556 |     n =
28004
         within  |             1.137315  -59.71419   94.73025 |     T =
18


. xtnbreg Firm_Births $base_vars $spatial_vars $demographic_vars $year_vars,
re exposure(Acres)

Fitting negative binomial (constant dispersion) model:

Iteration 0:   log likelihood =  -51795942  (not concave)
Iteration 1:   log likelihood =  -49724759  (not concave)
Iteration 2:   log likelihood =  -48730264  (not concave)
Iteration 3:   log likelihood =  -47170895  (not concave)
Iteration 4:   log likelihood =  -45963320  (not concave)
Iteration 5:   log likelihood =  -45021992  (not concave)

....

Iteration 9255:log likelihood = -470774.97  (not concave) Iteration 9256:log
likelihood = -470671.29  (not concave) Iteration 9257:log likelihood =
-470567.37 Iteration 9258:log likelihood = -402031.39  (backed up) Iteration
9259:log likelihood =  -388353.2 Iteration 9260:log likelihood =  -387803.5
Iteration 9261:log likelihood = -387800.57 Iteration 9262:log likelihood =
-387800.57

Iteration 0:   log likelihood = -475132.13
Iteration 1:   log likelihood = -458154.04
Iteration 2:   log likelihood = -457787.12
Iteration 3:   log likelihood =  -457787.1

Iteration 0:   log likelihood = -337007.31
Iteration 1:   log likelihood = -305214.12
Iteration 2:   log likelihood = -303965.91
Iteration 3:   log likelihood = -303963.67
Iteration 4:   log likelihood = -303963.67

Fitting full model:

Iteration 0:   log likelihood = -261858.51
Iteration 1:   log likelihood = -240203.89
Iteration 2:   log likelihood = -238845.87
Iteration 3:   log likelihood = -238800.18
Iteration 4:   log likelihood = -238800.08
Iteration 5:   log likelihood = -238800.08

Random-effects negative binomial regression     Number of obs      =
504072
Group variable: FIPSblock                       Number of groups   =
28004

Random effects u_i ~ Beta                       Obs per group: min =
18
                                                               avg =
18.0
                                                               max =
18

                                                Wald chi2(34)      =
57551.00
Log likelihood  = -238800.08                    Prob > chi2        =
0.0000

----------------------------------------------------------------------------
-----------------
                Firm_Births |      Coef.   Std. Err.      z    P>|z|
  [95% Conf. Interval]
----------------------------+-------------------------------------------
----------------------------+---------------------
...
                      _cons |  -2.502535   .0503268   -49.73   0.000
 -2.601174   -2.403897
                  ln(Acres) |          1  (exposure)
----------------------------+-------------------------------------------
----------------------------+---------------------
                      /ln_r |   2.151578   .0221937
  2.108079    2.195077
                      /ln_s |  -.3551327   .0117288
 -.3781207   -.3321448
----------------------------+-------------------------------------------
----------------------------+---------------------
                          r |   8.598419   .1908308
  8.232415    8.980694
                          s |   .7010804   .0082228
  .6851478    .7173835
----------------------------------------------------------------------------
-----------------
Likelihood-ratio test vs. pooled: chibar2(01) =  1.3e+05 Prob>=chibar2 =
0.000
(est1 stored)

. xtnbreg Firm_Births $base_vars $spatial_vars $demographic_vars $year_vars,
re exposure(Acres) iterate(100)

Fitting negative binomial (constant dispersion) model:

Iteration 0:   log likelihood =  -51795942  (not concave)
Iteration 1:   log likelihood =  -49724759  (not concave)
Iteration 2:   log likelihood =  -48730264  (not concave)
Iteration 3:   log likelihood =  -47170895  (not concave)
Iteration 4:   log likelihood =  -45963320  (not concave)
Iteration 5:   log likelihood =  -45021992  (not concave)

....

Iteration 97:  log likelihood = -6911325.4  (not concave) Iteration 98:  log
likelihood = -6837625.2  (not concave) Iteration 99:  log likelihood =
-6480527.6  (not concave) Iteration 100: log likelihood = -6375502.1  (not
concave) convergence not achieved

Iteration 0:   log likelihood = -475132.13
Iteration 1:   log likelihood = -458154.04
Iteration 2:   log likelihood = -457787.12
Iteration 3:   log likelihood =  -457787.1

Iteration 0:   log likelihood =  -457787.1  (not concave)
Iteration 1:   log likelihood = -447149.68  (not concave)
Iteration 2:   log likelihood = -433018.99
Iteration 3:   log likelihood = -374195.22
Iteration 4:   log likelihood = -343626.28
Iteration 5:   log likelihood = -325391.13
Iteration 6:   log likelihood = -308563.02
Iteration 7:   log likelihood = -304292.41
Iteration 8:   log likelihood = -303973.76
Iteration 9:   log likelihood = -303963.67
Iteration 10:  log likelihood = -303963.67

Fitting full model:

Iteration 0:   log likelihood = -261858.51
Iteration 1:   log likelihood = -240203.89
Iteration 2:   log likelihood = -238845.87
Iteration 3:   log likelihood = -238800.18
Iteration 4:   log likelihood = -238800.08
Iteration 5:   log likelihood = -238800.08

Random-effects negative binomial regression     Number of obs      =
504072
Group variable: FIPSblock                       Number of groups   =
28004

Random effects u_i ~ Beta                       Obs per group: min =
18
                                                               avg =
18.0
                                                               max =
18

                                                Wald chi2(34)      =
57551.00
Log likelihood  = -238800.08                    Prob > chi2        =
0.0000

----------------------------------------------------------------------------
-----------------
                Firm_Births |      Coef.   Std. Err.      z    P>|z|
  [95% Conf. Interval]
----------------------------+-------------------------------------------
----------------------------+---------------------
...
                      _cons |  -2.502535   .0503268   -49.73   0.000
 -2.601174   -2.403897
                  ln(Acres) |          1  (exposure)
----------------------------+-------------------------------------------
----------------------------+---------------------
                      /ln_r |   2.151578   .0221937
  2.108079    2.195077
                      /ln_s |  -.3551327   .0117288
 -.3781207   -.3321448
----------------------------+-------------------------------------------
----------------------------+---------------------
                          r |   8.598419   .1908308
  8.232415    8.980694
                          s |   .7010804   .0082228
  .6851478    .7173835
----------------------------------------------------------------------------
-----------------
Likelihood-ratio test vs. pooled: chibar2(01) =  1.3e+05 Prob>=chibar2 =
0.000
(est1 stored)

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index