RE: st: Multinomial Logit Model - Cramer Ridder Test

[Nick Cox <n.j.cox@durham.ac.uk>]

My guess is that you are calling -mlogit- under version control, i.e. the initial -version 7-  is biting you. 
It may be that changing that to -version 11- would be enough to fix the problem, but if you do that _you take full responsibility_, not me, nor the author of -crtest-! 

Despite its summary, the FAQ 

FAQ     . . . . . . . . . . . . . . .  User-written programs and Stata version
        . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  N. J. Cox
        7/11    I have a certain version of Stata and have come
                across a user-written program that is apparently
                written for a later version of Stata. What are
                my options?
                http://www.stata.com/support/faqs/lang/version2.html

is relevant to you in saying more about -version-. 

Nick 
n.j.cox@durham.ac.uk 

Ian Li

Nick, the output from -trace- is copied below.

- version 7.0
- tempvar sample tmp new
- tempname b valnum rcount nxtrow coef output C testn
- if "`e(cmd)'" != "mlogit" {
= if "mlogit" != "mlogit" {
  di _n in y "crtest" in r " only works after " in y "mlogit"
  exit 498
  }
- local depvar = e(depvar)
- matrix `coef' = e(b)
= matrix __000007 = e(b)
- local cols = colsof(`coef')
= local cols = colsof(__000007)
- local regs = `cols' / (e(k_out) - 1)
= local regs = 28 / (e(k_out) - 1)
- matrix `b' = `coef'[1, 1..`regs']
= matrix __000003 = __000007[1, 1..4.666666666666667]
- local rhs : colnames(`b')
= local rhs : colnames(__000003)
- local rhs : subinstr local rhs "_cons" ""
- _pecats
- local catnms8 "`r(catnms8)'"
= local catnms8 "1 2 3 4 5 6 7"
- local ll = e(ll)
- local numcasts = e(k_out)
- local df = e(df_m) / (e(k_out) - 1)
- if "`e(wtype)'" != "" {
= if "" != "" {
  local wtis "[`e(wtype)'`e(wexp)']"
  local wexp2 "`e(wexp)'"
  }
- quietly {
- generate `sample' = e(sample)
= generate __000000 = e(sample)
- if "`e(wtype)'" == "" | "`e(wtype)'" == "aweight" | "`e(wtype)'" == "pweight" {
= if "" == "" | "" == "aweight" | "" == "pweight" {
- count if `sample'
= count if __000000
- scalar `testn' = r(N)
= scalar __00000A = r(N)
- }
- else if "`e(wtype)'" == "fweight" | "`e(wtype)'" == "iweight" {
= else if "" == "fweight" | "" == "iweight" {
  local wtexp = substr("`e(wexp)'", 3, .)
  gen `tmp' = (`wtexp') * `sample'
  su `tmp', meanonly
  scalar `testn' = round(r(sum),1)
  }
- }
- if e(N) ~= `testn' {
= if e(N) ~= __00000A {
  di _n in r "data has been altered since " in y "mlogit" in r " was estimated"
  exit 459
  }
- if e(k_out) == 2 {
  di _n in r "Cramer-Ridder test requires at least 3 dependent categories"
  exit 148
  }
- mat `output' = (1, 1, 1, 1)
= mat __000008 = (1, 1, 1, 1)
- mat colnames `output' = "ln L" "ln Lr" LR "P>chi2"
= mat colnames __000008 = "ln L" "ln Lr" LR "P>chi2"
- qui tabulate `depvar' if `sample', matrow(`valnum') matcell(`rcount')
= qui tabulate overed if __000000, matrow(__000004) matcell(__000005)
- local nrows = rowsof(`valnum')
= local nrows = rowsof(__000004)
- local c = `valnum'[1,1]
= local c = __000004[1,1]
- forval i = 2 / `nrows' {
= forval i = 2 / 7 {
- forval i = 2 / `nrows' {
- local c = max(`c', `valnum'[`i',1])
= local c = max(1, __000004[2,1])
- }
- local c = max(`c', `valnum'[`i',1])
= local c = max(2, __000004[3,1])
- }
- local c = max(`c', `valnum'[`i',1])
= local c = max(3, __000004[4,1])
- }
- local c = max(`c', `valnum'[`i',1])
= local c = max(4, __000004[5,1])
- }
- local c = max(`c', `valnum'[`i',1])
= local c = max(5, __000004[6,1])
- }
- local c = max(`c', `valnum'[`i',1])
= local c = max(6, __000004[7,1])
- }
- if "`e(wtype)'" == "" {
= if "" == "" {
- gen `new' = 0
= gen __000002 = 0
- quietly forval count1 = 1 / `numcasts' {
= quietly forval count1 = 1 / 7 {
- quietly forval count1 = 1 / `numcasts' {
- tempname ccount1
- local c2 = `count1' + 1
= local c2 = 1 + 1
- forval count2 = `c2' / `numcasts' {
= forval count2 = 2 / 7 {
- forval count2 = `c2' / `numcasts' {
- tempname ccount2 adj_`count1'_`count2'
= tempname ccount2 adj_1_2
- tempname LR_`count1'_`count2'
= tempname LR_1_2
- replace `new' = `depvar'
= replace __000002 = overed
- replace `new' = `c' + 1 if `depvar' == `valnum'[`count1', 1] | `depvar' == `valnum'[`c
> ount2', 1]
= replace __000002 = 7 + 1 if overed == __000004[1, 1] | overed == __000004[2, 1]
- mlogit `new' `rhs' if `sample'
= mlogit __000002 o.female o.noneng o.nonaust o. female if __000000
factor variables and time-series operators not allowed
  local llp = e(ll)
  scalar `ccount1' = `rcount'[`count1', 1]
  scalar `ccount2' = `rcount'[`count2', 1]
  scalar `adj_`count1'_`count2'' = `ccount1' * ln(`ccount1') + `ccount2' * ln(`ccount2')
>  -(`ccount1' + `ccount2') * ln(`ccount1'+ `ccount2')
  local llr = `llp' + `adj_`count1'_`count2''
  scalar `LR_`count1'_`count2'' = 2 * (`ll' - `llr')
  local chi2 = `LR_`count1'_`count2''
  local pval = chi2tail(`df', `LR_`count1'_`count2'')
  local s1`count1'_`count2' : word `count1' of `catnms8'
  local s2`count1'_`count2' : word `count2' of `catnms8'
  mat `nxtrow' = (`ll', `llr', `chi2', `pval')
  mat roweq `nxtrow' = "`s1`count1'_`count2''"
  mat rownames `nxtrow' = "`s2`count1'_`count2''"
  mat `output' = `output' \ `nxtrow'
  }
  }
  di _n in g "**** Cramer-Ridder test for combining outcome categories"
  di _n in g "Ho: Candidates for pooling have the same regressor coefficients "
  di in g "    apart from the intercept"
  matrix `C' = `output'[2..rowsof(`output'),1..4]
  matrix list `C', format(%9.3f) noheader
  di _n in g "degrees of freedom for chi-square distribution:  " in w `df'
  }
------------------------------------------------------------------------- end crtest ---
r(101);

Richard's solution of using -mlogtest- 'works', in the sense that no error messages are generated. However, as Richard mentioned, he does not know how the test for combining outcomes in -mlogtest- differs with -crtest-. I'll think on it and see if -mlogtest- would suit my purpose. 

Regards
Ian

----------------------------------------
> From: n.j.cox@durham.ac.uk
> To: statalist@hsphsun2.harvard.edu
> Date: Mon, 14 Nov 2011 10:35:34 +0000
> Subject: RE: st: Multinomial Logit Model - Cramer Ridder Test
>
> -crtest- as cited here long predates the introduction of factor variables in Stata. But you are not explicitly using factor variable notation. So it could well be something else. I'd
>
> . set trace on
> . set traced 1
> . crtest
>
> -- so you can report what is issuing the error.
>
> Nick
> n.j.cox@durham.ac.uk
>
> Muhammad Anees
>
> try without the -if year == 2009- as the help file suggests -crtest-
> "performs the test for pooling states" after the Multinomial logit
> models. The error suggest to not use time operators and factor
> variables. Also read the -help crtest- file for more information.
>
> On Mon, Nov 14, 2011 at 11:18 AM, Ian Li <ianz.magic@hotmail.com> wrote:
>
> > I am using Stata 11.2 on a Windows OS.
> >
> > I am estimating a multinomial logit model with 7 categorical outcomes. As the estimated log odds ratio and marginal effects for some of the outcomes (e.g.) are insignificant in statistical and economic terms, I would like to do a Cramer Ridder test (see Cramer J.S. and Ridder G. (1991), "Pooling States in the Multinomial Logit Model", Journal of Econometrics, 47, 267-272).
> >
> > There is a user-written program for this test:
> >
> > package crtest from http://fmwww.bc.edu/RePEc/bocode/c
> >
> >       Distribution-Date: 20031203
> >
> >       Author:  Joao Pedro Azevedo, University of Newcastle-upon-Tyne, UK
> >       Support: email j.p.azevedo@ncl.ac.uk
> >
> > However, when I estimate the logit model and use the test I get the following output and error message:
> >
> >
> > . mlogit overed  female noneng nonaust if year == 2009, vce (robust)
> >
> > Iteration 0:   log pseudolikelihood = -109594.56
> > Iteration 1:   log pseudolikelihood = -107547.29
> > Iteration 2:   log pseudolikelihood = -106989.03
> > Iteration 3:   log pseudolikelihood = -106976.66
> > Iteration 4:   log pseudolikelihood = -106976.62
> > Iteration 5:   log pseudolikelihood = -106976.62
> >
> > Multinomial logistic regression                   Number of obs   =      68352
> >                                                   Wald chi2(18)   =    5211.03
> >                                                   Prob > chi2     =     0.0000
> > Log pseudolikelihood = -106976.62                 Pseudo R2       =     0.0239
> >
> > ------------------------------------------------------------------------------
> >              |               Robust
> >       overed |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
> > -------------+----------------------------------------------------------------
> > 1            |  (base outcome)
> > -------------+----------------------------------------------------------------
> > 2            |
> >       female |   .0054676   .0238051     0.23   0.818    -.0411895    .0521248
> >       noneng |  -.1713411    .036681    -4.67   0.000    -.2432346   -.0994477
> >      nonaust |  -.4023007   .0666538    -6.04   0.000    -.5329397   -.2716617
> >        _cons |  -.6301703   .0196192   -32.12   0.000    -.6686234   -.5917173
> > -------------+----------------------------------------------------------------
> > 3            |
> >       female |  -.2061721   .0220245    -9.36   0.000    -.2493394   -.1630049
> >       noneng |   .2473055   .0309354     7.99   0.000     .1866732    .3079377
> >      nonaust |   .7666135   .0449734    17.05   0.000     .6784672    .8547598
> >        _cons |  -.4356624   .0180227   -24.17   0.000    -.4709862   -.4003386
> > -------------+----------------------------------------------------------------
> > 4            |
> >       female |  -.1237307   .0222175    -5.57   0.000    -.1672762   -.0801853
> >       noneng |   .1045945   .0315106     3.32   0.001     .0428348    .1663542
> >      nonaust |   .6037373   .0464335    13.00   0.000     .5127293    .6947453
> >        _cons |  -.4583203   .0183183   -25.02   0.000    -.4942236    -.422417
> > -------------+----------------------------------------------------------------
> > 5            |
> >       female |  -.2056775   .0364563    -5.64   0.000    -.2771306   -.1342244
> >       noneng |   .2566406   .0474816     5.41   0.000     .1635783    .3497029
> >      nonaust |   1.251198   .0592131    21.13   0.000     1.135142    1.367253
> >        _cons |  -1.871285   .0300228   -62.33   0.000    -1.930128   -1.812441
> > -------------+----------------------------------------------------------------
> > 6            |
> >       female |  -.4014462   .0451968    -8.88   0.000    -.4900303   -.3128621
> >       noneng |   1.138732   .0587742    19.37   0.000     1.023536    1.253927
> >      nonaust |   2.322202   .0632143    36.74   0.000     2.198304      2.4461
> >        _cons |  -2.876826   .0415611   -69.22   0.000    -2.958284   -2.795368
> > -------------+----------------------------------------------------------------
> > 7            |
> >       female |  -.0626167   .1889866    -0.33   0.740    -.4330237    .3077902
> >       noneng |  -.1017222   .2996702    -0.34   0.734    -.6890651    .4856207
> >      nonaust |   .9465996   .3576756     2.65   0.008     .2455683    1.647631
> >        _cons |  -5.264885   .1549144   -33.99   0.000    -5.568511   -4.961258
> > ------------------------------------------------------------------------------
> >
> > .
> > end of do-file
> >
> > . crtest
> > factor variables and time-series operators not allowed
> > r(101);
> >
> > Could anyone please advise on what is causing the error message, and how I may resolve thi?
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/