Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: xtmixed with nonrtolerance. What happens?


From   "Lukas Bösch" <[email protected]>
To   [email protected]
Subject   Re: st: xtmixed with nonrtolerance. What happens?
Date   Thu, 23 Jun 2011 22:40:38 +0200

Because i didnt transform the year and the export, named as quantity, into z-scores they kept their original names in the first models. 
I just did the transformation and ran the model again, but it still doesnt converge, however seems to work a little better.

. xtmixed centquantity2 centyear2 centforestarea2 centgdp2 centlandarea2 centpopulation2|| _all: R.country || _all: R.genus

Performing EM optimization: 
Performing gradient-based optimization: 

Iteration 0:   log restricted-likelihood = -4875.1075  
Iteration 1:   log restricted-likelihood = -4870.6476  
Iteration 2:   log restricted-likelihood = -4870.5095  
Iteration 3:   log restricted-likelihood = -4870.4438  
Iteration 4:   log restricted-likelihood = -4870.4118  (backed up)
Iteration 5:   log restricted-likelihood = -4870.4039  (backed up)
Iteration 6:   log restricted-likelihood = -4870.3999  (backed up)
Iteration 7:   log restricted-likelihood = -4870.3979  (backed up)
Iteration 8:   log restricted-likelihood = -4870.3969  (backed up)
Iteration 9:   log restricted-likelihood = -4870.3967  (backed up)
numerical derivatives are approximate
nearby values are missing
Iteration 10:  log restricted-likelihood = -4870.3966  (backed up)
numerical derivatives are approximate
nearby values are missing
Iteration 11:  log restricted-likelihood = -4870.3966  (backed up)
numerical derivatives are approximate
nearby values are missing
numerical derivatives are approximate
nearby values are missing
Hessian has become unstable or asymmetric

Mixed-effects REML regression                   Number of obs      =      6192
Group variable: _all                            Number of groups   =         1
                                                Obs per group: min =      6192
                                                               avg =    6192.0
                                                               max =      6192
                                                Wald chi2(5)       =      9.26
Log restricted-likelihood = -4875.1075          Prob > chi2        =    0.0991

centquanti~2 |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
   centyear2 |  -.0169763    .008528    -1.99   0.047     -.033691   -.0002616
centfores~a2 |  -.0846178   .0568262    -1.49   0.136    -.1959951    .0267595
    centgdp2 |  -.0173484   .0354612    -0.49   0.625    -.0868509    .0521542
centlandar~2 |  -.4531947   .5468347    -0.83   0.407    -1.524971    .6185816
centpopul~n2 |   .1910553   .0876979     2.18   0.029     .0191707      .36294
       _cons |   .2434439   .4596746     0.53   0.596    -.6575018     1.14439

  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
_all: Identity               |
               sd(R.country) |   2.684813          .             
_all: Identity               |
                 sd(R.genus) |   .0579011          .             
                sd(Residual) |   .5155702          .             
LR test vs. linear regression:       chi2(2) =  7810.42   Prob > chi2 = 0.0000

Note: LR test is conservative and provided only for reference.
Warning: convergence not achieved; estimates are based on iterated EM



Here the summarize output of all the variables:

. sum centquantity2

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
centquanti~2 |      6192    2.17e-09           1  -.0732263    34.3665

. sum centyear2

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
   centyear2 |      6192           0    1.000024  -1.626886   1.626886

. sum centforestarea2

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
centfores~a2 |      6192   -.0043667     1.00682  -2.396995   2.746216

. sum centgdp2

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
    centgdp2 |      6192   -.0835699    .8318088  -.3333735   5.257175

. sum centlandarea2

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
centlandar~2 |      6192   -.0336882    .9528875  -.6987395   2.490177

. sum centpopulation2

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
centpopul~n2 |      6192   -.0018452    1.069818  -.6711841   8.741787

At last a short extract of the dataset, with the original quantity, in order to see the structure:

quantity country genus	 centyear2   centagriculturalland2
0	 USA	Tursiops 1.19305	.6379885
0	 USA	Tursiops 1.409968	.6239355
6.08	 USA	Tursiops 1.626886	.6126072
40.29	 USA	Ursus	-1.626886	.7022238
65.375	 USA	Ursus	-1.409968	.7022238
140.255	 USA	Ursus	-1.19305	.6926397

In total 40 countries, 213 genus and 21 independendt variables over a period of 16 years with 6192 observations. As there is no hirarchical structure, there are no different levels. There is one level for the quantity exported and two randome effects at this level, the country and genus.
After having transformed the quantity to z-scores, would you still recommend dividing it by 100k?

Thank you

Lukas


-------- Original-Nachricht --------
> Datum: Thu, 23 Jun 2011 14:41:44 -0400
> Von: Joerg Luedicke <[email protected]>
> An: [email protected]
> Betreff: Re: st: xtmixed with nonrtolerance. What happens?

> k stands for 1000 (as in kb=1000 bytes, for instance). What are your
> Level 1 observations (i.e., the  6192)? If only 72 bears were exported
> from the US in a given year then figures in the ballpark of hundreds
> of thousands appear fairly high to me?
> 
> J.
> 
> On Thu, Jun 23, 2011 at 2:14 PM, "Lukas Bösch" <[email protected]> wrote:
> > In my opinion the scales dont differ wildly.
> > I am not a statistician though, so maybe you have a different opinion.
> >
> >
> > . sum centgdp2
> >
> >    Variable |       Obs        Mean    Std. Dev.       Min
>        Max
> > -------------+--------------------------------------------------------
> >    centgdp2 |      6192   -.0835699    .8318088  -.3333735  
> 5.257175
> >
> > . sum centlandarea2
> >
> >    Variable |       Obs        Mean    Std. Dev.       Min
>        Max
> > -------------+--------------------------------------------------------
> > centlandar~2 |      6192   -.0336882    .9528875  -.6987395  
> 2.490177
> >
> > . sum centpopulation2
> >
> >    Variable |       Obs        Mean    Std. Dev.       Min
>        Max
> > -------------+--------------------------------------------------------
> > centpopul~n2 |      6192   -.0018452    1.069818  -.6711841  
> 8.741787
> >
> > . sum centyear2
> >
> >    Variable |       Obs        Mean    Std. Dev.       Min
>        Max
> > -------------+--------------------------------------------------------
> >   centyear2 |      6192           0    1.000024  -1.626886
>   1.626886
> >
> > . sum centforestarea2
> >
> >    Variable |       Obs        Mean    Std. Dev.       Min
>        Max
> > -------------+--------------------------------------------------------
> > centfores~a2 |      6192   -.0043667     1.00682 -2.396995  
> 2.746216
> >
> > The dependent variable is export. The export of wild animal and plant
> products from one country to the rest of the world. For example: US export of
> Bears in 1992: 72.
> > Because I cannot sum up the export of different species to one export
> figure, obviously bears and pearls are not the same, i have to deal with
> those mixed models. Socioeconomic factors are set as fixed effects and the
> genus and countries as the variable effects.
> > As one species can be exported by different countries, the data is not
> hierarchic and country and genus are cross-classified. Or i think this is
> what it means. Two random effects at the same level for all observations.
> Joerge, can you explain what you mean with dividing by 100k? What does the k
> stand for?
> >
> > Thank you
> >
> > Lukas
> >
> > mixed modells-------- Original-Nachricht --------
> >> Datum: Thu, 23 Jun 2011 09:47:55 -0400
> >> Von: Joerg Luedicke <[email protected]>
> >> An: [email protected]
> >> Betreff: Re: st: xtmixed with nonrtolerance. What happens?
> >
> >> Your model did not converge using the default convergence criteria and
> >> with -nonrtolerance- you just turned off that default criteria
> >> (though, I do not know what criteria is used instead?). However, you
> >> should be very cautious with regard to the results.
> >>
> >> What is your dependent variable? From your output I gather that its
> >> predicted mean is roughly 900k at average values of your covariates.
> >> Maybe you should transform your dependent variable and fit the model
> >> again (e.g., dividing it by 100k).
> >>
> >> A question in regards to your random effects: are -country- and
> >> -genus- cross-classified?
> >>
> >> J.
> >>
> >> On Thu, Jun 23, 2011 at 6:21 AM, "Lukas Bösch" <[email protected]>
> wrote:
> >> > I transformed the data to z-scores (score-mean/stdeviation) before
> doing
> >> the regression.
> >> > What do you mean with differing scales? I have either percents, for
> >> example % forest area, or absolute figures, for example land area, in
> my
> >> dataset, but they are all transformed and should therefore be uniform.
> >> > What about nonrtolerance?
> >> >
> >> > Thank you
> >> >
> >> > Lukas
> >> >
> >> > -------- Original-Nachricht --------
> >> >> Datum: Wed, 22 Jun 2011 18:48:22 -0400
> >> >> Von: Stas Kolenikov <[email protected]>
> >> >> An: [email protected]
> >> >> Betreff: Re: st: xtmixed with nonrtolerance. What happens?
> >> >
> >> >> It looks like you have data with wildly differing scales. I
> understand
> >> >> that you need to interpret the results in the original scales, but
> >> >> maybe you could rescale your variables so that all of your
> >> >> coefficients would be about 1. Whether that will help convergence is
> >> >> anybody's telling, of course, but usually differences in the scales
> >> >> (and hence coefficients) of the order of 1e3-1e4 are detrimental to
> >> >> numeric convergence.
> >> >>
> >> >> On Wed, Jun 22, 2011 at 4:33 PM, "Lukas Bösch" <[email protected]>
> >> wrote:
> >> >> > Dear Statalist community.
> >> >> >
> >> >> > I am using Stata 10.0 and doing a mixed model analysis of export
> >> data.
> >> >> > After trying different options and always having trouble to get a
> >> >> propper output i finally found a way to get to my results. I however
> >> could not
> >> >> find any information about why it works and if it is allright. But
> let
> >> us
> >> >> first start with the problem:
> >> >> >
> >> >> > 1) This is the command i enter and the output stata creates:
> >> >> >
> >> >> > xtmixed quantity year centforestarea2 centgdp2 centlandarea2
> >> >> centpopulation2 || _all: R.country || _all: R.genus
> >> >> >
> >> >> > Performing EM optimization:
> >> >> >
> >> >> > Performing gradient-based optimization:
> >> >> >
> >> >> > Iteration 0:   log restricted-likelihood = -77051.164
> >> >> > Iteration 1:   log restricted-likelihood = -77046.704
> >> >> > Iteration 2:   log restricted-likelihood = -77046.565
> >> >> > Iteration 3:   log restricted-likelihood =   -77046.5
> >> >> > Iteration 4:   log restricted-likelihood = -77046.468  (backed
> up)
> >> >> > Iteration 5:   log restricted-likelihood =  -77046.46  (backed
> up)
> >> >> > Iteration 6:   log restricted-likelihood = -77046.456  (backed
> up)
> >> >> > Iteration 7:   log restricted-likelihood = -77046.454  (backed
> up)
> >> >> > numerical derivatives are approximate
> >> >> > nearby values are missing
> >> >> > Iteration 8:   log restricted-likelihood = -77046.453  (backed
> up)
> >> >> > numerical derivatives are approximate
> >> >> > nearby values are missing
> >> >> > Hessian has become unstable or asymmetric
> >> >> >
> >> >> > Mixed-effects REML regression                   Number of
> >> obs
> >> >>      =      6192
> >> >> > Group variable: _all                          
>  Number
> >> of
> >> >> groups   =         1
> >> >> >
> >> >> >
> >> >>  Obs per group: min =      6192
> >> >> >
> >> >>               avg =    6192.0
> >> >> >
> >> >>               max =      6192
> >> >> >
> >> >>  Wald chi2(5)       =      9.26
> >> >> > Log restricted-likelihood = -77051.164          Prob > chi2
> >> >>    =    0.0991
> >> >> >    quantity |      Coef.   Std. Err.      z    P>|z|
> >> >> [95% Conf. Interval]
> >> >> >        year |  -429.7599   215.8898    -1.99   0.047
> >> >>  -852.8961   -6.623654
> >> >> > centfores~a2 |  -9875.264   6631.861    -1.49   0.136
> >> >>  -22873.47    3122.945
> >> >> >    centgdp2 |  -2024.629   4138.469    -0.49   0.625
> >> >>  -10135.88    6086.621
> >> >> > centlandar~2 |  -52889.76   63817.96    -0.83   0.407
> >> >>  -177970.7    72191.13
> >> >> > centpopul~n2 |   22296.98   10234.72     2.18   0.029
> >> >> 2237.304    42356.66
> >> >> >       _cons |   895402.2   433369.4     2.07   0.039
> >> >> 46013.74     1744791
> >> >> >
> >> >> >  Random-effects Parameters  |   Estimate   Std. Err.    
> [95%
> >> >> Conf. Interval]
> >> >> >
> >> >> > _all: Identity               |
> >> >> >               sd(R.country) |   313329.2          .
> >> >> > _all: Identity               |
> >> >> >                 sd(R.genus) |   6757.304          .
> >> >> >                sd(Residual) |   60169.26          .
> >> >> > LR test vs. linear regression:       chi2(2) =  7810.42  
> Prob >
> >> >> chi2 = 0.0000
> >> >> >
> >> >> > Note: LR test is conservative and provided only for reference.
> >> >> > Warning: convergence not achieved; estimates are based on iterated
> EM
> >> >> >
> >> >> > Obviously Stata has a problem and can't calculate the standard
> errors
> >> of
> >> >> the random factors.
> >> >> >
> >> >> > 2) With the option nonrtolerance it works however:
> >> >> >
> >> >> > xtmixed quantity year centforestarea2 centgdp2 centlandarea2
> >> >> centpopulation2 || _all: R.country || _all: R.genus, nonrtolerance
> >> >> >
> >> >> > Performing EM optimization:
> >> >> >
> >> >> > Performing gradient-based optimization:
> >> >> >
> >> >> > Iteration 0:   log restricted-likelihood = -77051.164
> >> >> > Iteration 1:   log restricted-likelihood = -77046.704
> >> >> > Iteration 2:   log restricted-likelihood = -77046.565
> >> >> > Iteration 3:   log restricted-likelihood =   -77046.5
> >> >> > Iteration 4:   log restricted-likelihood = -77046.468  (backed
> up)
> >> >> > Iteration 5:   log restricted-likelihood =  -77046.46  (backed
> up)
> >> >> > Iteration 6:   log restricted-likelihood = -77046.456  (backed
> up)
> >> >> >
> >> >> > Computing standard errors:
> >> >> >
> >> >> > Mixed-effects REML regression                   Number of
> >> obs
> >> >>      =      6192
> >> >> > Group variable: _all                          
>  Number
> >> of
> >> >> groups   =         1
> >> >> >
> >> >> >
> >> >>  Obs per group: min =      6192
> >> >> >
> >> >>               avg =    6192.0
> >> >> >
> >> >>               max =      6192
> >> >> >
> >> >> >
> >> >> >
> >> >>  Wald chi2(5)       =      9.22
> >> >> > Log restricted-likelihood = -77046.456          Prob > chi2
> >> >>    =    0.1008
> >> >> >    quantity |      Coef.   Std. Err.      z    P>|z|
> >> >> [95% Conf. Interval]
> >> >> >        year |  -429.7645   216.4073    -1.99   0.047
> >> >> -853.915   -5.614053
> >> >> > centfores~a2 |  -9885.307    6647.52    -1.49   0.137
> >> >>  -22914.21    3143.592
> >> >> >    centgdp2 |  -2021.312   4148.464    -0.49   0.626
> >> >>  -10152.15    6109.527
> >> >> > centlandar~2 |  -52859.75   63778.66    -0.83   0.407
> >> >>  -177863.6    72144.12
> >> >> > centpopul~n2 |   22276.96   10257.46     2.17   0.030
> >> >> 2172.715     42381.2
> >> >> >       _cons |   895338.1   434389.3     2.06   0.039
> >> >> 43950.68     1746726
> >> >> >
> >> >> >  Random-effects Parameters  |   Estimate   Std. Err.    
> [95%
> >> >> Conf. Interval]
> >> >> > _all: Identity               |
> >> >> >               sd(R.country) |   313133.2    36075.6
> >> >>  249840.9    392459.4
> >> >> > _all: Identity               |
> >> >> >                 sd(R.genus) |   3440.288   1355.694
> >> >>  1589.157    7447.712
> >> >> >                sd(Residual) |   60315.87   545.9681
> >> >>  59255.23     61395.5
> >> >> > LR test vs. linear regression:       chi2(2) =  7819.83  
> Prob >
> >> >> chi2 = 0.0000
> >> >> > Note: LR test is conservative and provided only for reference.
> >> >> >
> >> >> > Can someone explain to me why it works with nonrtolerance and tell
> me
> >> if
> >> >> these outputs are as reliable as if they were created without
> >> >> nonrtolerance. I searched in the stata help and on stata.com but
> could
> >> not find more
> >> >> information about this.
> >> >> >
> >> >> > Kind regards
> >> >> >
> >> >> > Lukas
> >> >> >
> >> >> > --
> >> >> > NEU: FreePhone - kostenlos mobil telefonieren!
> >> >> > Jetzt informieren: http://www.gmx.net/de/go/freephone
> >> >> > *
> >> >> > *   For searches and help try:
> >> >> > *   http://www.stata.com/help.cgi?search
> >> >> > *   http://www.stata.com/support/statalist/faq
> >> >> > *   http://www.ats.ucla.edu/stat/stata/
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Stas Kolenikov, also found at http://stas.kolenikov.name
> >> >> Small print: I use this email account for mailing lists only.
> >> >>
> >> >> *
> >> >> *   For searches and help try:
> >> >> *   http://www.stata.com/help.cgi?search
> >> >> *   http://www.stata.com/support/statalist/faq
> >> >> *   http://www.ats.ucla.edu/stat/stata/
> >> >
> >> > --
> >> > NEU: FreePhone - kostenlos mobil telefonieren!
> >> > Jetzt informieren: http://www.gmx.net/de/go/freephone
> >> > *
> >> > *   For searches and help try:
> >> > *   http://www.stata.com/help.cgi?search
> >> > *   http://www.stata.com/support/statalist/faq
> >> > *   http://www.ats.ucla.edu/stat/stata/
> >> >
> >>
> >> *
> >> *   For searches and help try:
> >> *   http://www.stata.com/help.cgi?search
> >> *   http://www.stata.com/support/statalist/faq
> >> *   http://www.ats.ucla.edu/stat/stata/
> >
> > --
> > Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
> > belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
> >
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

-- 
NEU: FreePhone - kostenlos mobil telefonieren!			
Jetzt informieren: http://www.gmx.net/de/go/freephone
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index