st: Re: st: Re: st: BOOTSTRAP: the standard errors of marginal effects of MIXLOGIT‏

[Arne Risa Hole <arnehole@gmail.com>]

Dear Kayo,

You need to set this up similarly to the example in the tread I
referred to. Note in particular the following:

1) You need to tell -bootstrap- to draw groups of observations
relating to individuals with replacement, not single observations.
This is done using the -cluster()- option of bootstrap.

2) You need to ensure that proper ID variables are used when running
-mixlogit- on the bootstrap samples. This is done using the
-idcluster()- and -group()- options of bootstrap.

If you read "help bootstrap" carefully so that you understand the
meaning of these options and look at the example that I sent you again
I think you'll be able to fix the problem yourself.

Arne

On 22 June 2012 08:34, nagi kayo <kayonagi@hotmail.co.jp> wrote:
>
> Dear Professor Hole
>
>
>
> Thank you very much for your prompt reply. I greatly appreciate your help,
>
> and I am realy sorry for my delay in responding.
>
>
>
>> Your procedure is incorrect as the model needs to be re-estimated for
>> each bootstrap sample. In other words your “marginal_inc” program
>> should include the call to -mixlogit-.
>
>
>
> Based on your advice, I corrected my program as follows:
>
>
>
> cap program drop marginal_inc
> program marginal_inc, rclass
>  mixlogit d d1 d2 d3 d1inc d2inc d3inc, group(id) rand(p)
>  mixlpred prep_base
>  quietly replace d1inc=d1inc+1 if alt==1
>  quietly replace d2inc=d2inc+1 if alt==2
>  quietly replace d3inc=d3inc+1 if alt==3
>  quietly replace d4inc=d4inc+1 if alt==4
>  mixlpred prep_inc
>
>  quietly sum prep_base if alt==1
>  local av_prep_base1 = r(mean)
>  quietly sum prep_inc if alt==1
>  local av_prep_inc1 = r(mean)
>  return scalar marge_inc1 = `av_prep_inc1' - `av_prep_base1'
>  quietly sum prep_base if alt==2
>  local av_prep_base2 = r(mean)
>  quietly sum prep_inc if alt==2
>  local av_prep_inc2 = r(mean)
>  return scalar marge_inc2 = `av_prep_inc2' - `av_prep_base2'
>  quietly sum prep_base if alt==3
>  local av_prep_base3 = r(mean)
>  quietly sum prep_inc if alt==3
>  local av_prep_inc3 = r(mean)
>  return scalar marge_inc3 = `av_prep_inc3' - `av_prep_base3'
>  quietly sum prep_base if alt==4
>  local av_prep_base4 = r(mean)
>  quietly sum prep_inc if alt==4
>  local av_prep_inc4 = r(mean)
>  return scalar marge_inc4 = `av_prep_inc4' - `av_prep_base4'
> end
> bootstrap r(marge_inc1), reps(1000): marginal_inc
> bootstrap r(marge_inc2), reps(1000): marginal_inc
> bootstrap r(marge_inc3), reps(1000): marginal_inc
> bootstrap r(marge_inc4), reps(1000): marginal_inc
>
>
> However, STATA returned the follwing error message.
>
>
>
> insufficient observations to compute bootstrap standard errors
> no results will be saved
> r(2000);
>
>
> Could you please teach me which part of my program is incorrect?
>
> I am really sorry to trouble you so much.
>
>
>
> with my thanks and best wishes,
>
> Kayo
>
>
>
>
>
>
>
>
> ----------------------------------------
>> Date: Sun, 3 Jun 2012 13:09:57 +0100
>> Subject: st: Re: st: BOOTSTRAP: the standard errors of marginal effects of MIXLOGIT‏
>> From: arnehole@gmail.com
>> To: statalist@hsphsun2.harvard.edu
>>
>> Dear Kayo
>>
>> Your procedure is incorrect as the model needs to be re-estimated for
>> each bootstrap sample. In other words your “marginal_inc” program
>> should include the call to -mixlogit-. Whether this is practical or
>> not depends on how long it takes to estimate your model – you may be
>> in for a long wait!
>>
>> See this thread
>> <http://www.stata.com/statalist/archive/2010-11/msg01025.html> for an
>> example of how -bootstrap- can be used with -mixlogit-.
>>
>> Arne
>>
>> PS Note that you can bootstrap several statistics in one go – you
>> don’t need to run -bootstrap- for each marginal effect.
>>
>> On 3 June 2012 09:01, nagi kayo <kayonagi@hotmail.co.jp> wrote:
>> > Dear Professor Arne Risa Hole and all
>> >
>> > I estimated mixed logit model using the command -mixlogit- and
>> > calculated the marginal effects using the command -mixlpred-.
>> > (Once again, I greatly appreciate that professor Arne Risa Holl
>> >
>> > gave me advice on how to calculate the marginal effects using
>> >
>> > "mixlpred" last february)
>> >
>> > However, now I am having trouble obtaining the standard errors and
>> > p-value of marginal effects using the bootstrap.
>> >
>> > First, please let me explain the data.
>> > My data set is like below.
>> > (Actually, my data set inlude 10,000 id, but here I show you the data only on two id
>> > for simplicity. In addition, the data I used in the estimation include
>> >
>> > the data on many household characteristics such as age, wealth, education level.)
>> >
>> > id alt d d1 d2 d3 d1inc d2inc d3inc d4inc p
>> > 1 1 1 1 0 0 665 0 0 0 0.214
>> > 1 2 0 0 1 0 0 665 0 0 0.186
>> > 1 3 0 0 0 1 0 0 665 0 0.381
>> > 1 4 0 0 0 0 0 0 0 665 0.219
>> > 2 1 0 1 0 0 779 0 0 0 0.553
>> > 2 2 1 0 1 0 0 779 0 0 0.301
>> > 2 3 0 0 0 1 0 0 779 0 0.107
>> > 2 4 0 0 0 0 0 0 0 779 0.039
>> >
>> >
>> >
>> > id: households
>> > alt: 1=households are worried about their retirement life
>> > because the pension benefit is NOT enough.
>> > 2=households are worried about their retirement life
>> > for some reasons other than pension.
>> > 3=households are NOT worried about their retirement life
>> > because the pension benefit is enough.
>> > 4=households are NOT worried about their retirement life
>> > for some reasons other than pension.
>> > d: dummy which equals one if households choose "alt" in the same row.
>> > (so id 1 chose alt 1, and id 2 chose alt 2.)
>> > d1-d3: intercepts
>> > d1inc-d4inc: real households income
>> > p: an alternative specific variable
>> >
>> >
>> >
>> > using the above data, i did the estimation as follows:
>> >
>> >
>> >
>> > **********************************************************************
>> > mixlogit d d1 d2 d3 d1inc d2inc d3inc, group(id) rand(p)
>> > mixlpred prep_base
>> > preserve
>> > quietly replace d1inc=d1inc+1 if alt==1
>> > quietly replace d2inc=d2inc+1 if alt==2
>> > quietly replace d3inc=d3inc+1 if alt==3
>> > quietly replace d4inc=d4inc+1 if alt==4
>> > mixlpred prep_inc
>> > cap program drop marginal_inc
>> > program marginal_inc, rclass
>> > quietly sum prep_base if alt==1
>> > local av_prep_base1 = r(mean)
>> > quietly sum prep_inc if alt==1
>> > local av_prep_inc1 = r(mean)
>> > return scalar marge_inc1 = `av_prep_inc1' - `av_prep_base1'
>> > quietly sum prep_base if alt==2
>> > local av_prep_base2 = r(mean)
>> > quietly sum prep_inc if alt==2
>> > local av_prep_inc2 = r(mean)
>> > return scalar marge_inc2 = `av_prep_inc2' - `av_prep_base2'
>> > quietly sum prep_base if alt==3
>> > local av_prep_base3 = r(mean)
>> > quietly sum prep_inc if alt==3
>> > local av_prep_inc3 = r(mean)
>> > return scalar marge_inc3 = `av_prep_inc3' - `av_prep_base3'
>> > quietly sum prep_base if alt==4
>> > local av_prep_base4 = r(mean)
>> > quietly sum prep_inc if alt==4
>> > local av_prep_inc4 = r(mean)
>> > return scalar marge_inc4 = `av_prep_inc4' - `av_prep_base4'
>> > end
>> > bootstrap r(marge_inc1), reps(1000): marginal_inc
>> > bootstrap r(marge_inc2), reps(1000): marginal_inc
>> > bootstrap r(marge_inc3), reps(1000): marginal_inc
>> > bootstrap r(marge_inc4), reps(1000): marginal_inc
>> > restore
>> > ************************************************************************
>> >
>> >
>> > The results of bootstrap is as follows:
>> >
>> > Standard errors P-value
>> > marge_inc1 -0.0011 4.44e-06 0.000
>> > marge_inc2 -0.0008 2.70e-06 0.000
>> > marge_inc3 0.0013 5.41e-06 0.000
>> > marge_inc4 0.0007 3.36e-06 0.000
>> > ************************************************************************
>> >
>> >
>> > Apparently, there is no problem in the above results.
>> > However, all the standard errors of marginal effects of ANY explanatory variable
>> > (e.g. age, education level, risk aversion) are very small,
>> > and all p-values of marginal effects of ANY explanatory variables are 0.000.
>> > I am wondering if my program has something wrong.
>> >
>> > Could anyone give me any comments?
>> > I greatly appreciate it.
>> >
>> > with best wishes,
>> > KAYO
>> >
>> > *
>> > * For searches and help try:
>> > * http://www.stata.com/help.cgi?search
>> > * http://www.stata.com/support/statalist/faq
>> > * http://www.ats.ucla.edu/stat/stata/
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/statalist/faq
>> * http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/