Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Computation Speed for Bootstrapping


From   Danny Dan <danny2011dan@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Computation Speed for Bootstrapping
Date   Mon, 5 Dec 2011 16:24:39 -0600

Dear Ben,

Good news is I can now run my program as I have fixed the error. The
error r(2000) was coming up as while estimating my model one of the
variables was getting dropped from the data, which I later found and
fixed and now it is running without any problem and indeed Jeph's and
your's suggestions have worked and increased the speed of computation.

Thank you so very much to both of you. I appreciate your help.

Regards,

Dan

On Mon, Dec 5, 2011 at 4:05 AM, Benjamin Volland <volland@econ.mpg.de> wrote:
> Dan,
>
> without knowing your code it is hard to say what's going on. It is
> particularly puzzling because the only thing that (presumably) changes from
> your running code to Jeph's suggestion is that you specify the starting
> values for maximization.
>
> When running the following very simple example:
> ******************
> version 12
> set more off
> timer clear
>
> webuse grunfeld, clear
> egen minvest = mean(invest), by(company)
> gen binvest = 1 if invest > minvest
> replace binvest = 0 if invest <= minvest
> xtset company time
>
> timer on 1
> xtprobit binvest kstock mvalue
> matrix b = e(b)
> set seed 911
> xtprobit binvest kstock mvalue, vce(boot, reps(20)) from(b)
> timer off 1
>
> timer on 2
> set seed 911
> xtprobit binvest kstock mvalue, vce(boot, reps(20))
> timer off 2
>
> timer list
> ******************
> everything works fine on my machine. Moreover, -timer- reveals that Jeph's
> suggestion substantially increases estimation speed in this simple case.
> . timer list
>   1:      3.50 /        1 =       3.5000
>   2:      4.16 /        1 =       4.1560
>
> Another thought on the speed of your estimation: it may be an idea to check
> your independents for (multi)collinearity, as this may substantially slow
> down convergence.
>
> Best, Ben
>
>
>
> On 04/12/2011 05:05, Danny Dan wrote:
>>
>> Thank you so much Ben for replying to my query. I have tried
>> implementing your's and Jeph's suggestions but now it is giving me the
>> following error r(2000):
>>
>> insufficient observations to compute bootstrap standard errors
>> no results will be saved
>>
>> However, without those additional commands my bootstrap is running,
>> although the computation is extremely slow (taking almost 4 days for
>> only 50 reps. My data set contains 10,000 obs.).
>>
>> Also do you or any statalist users have any idea why the problem
>> r(2000) (insufficient observations to compute bootstrap standard
>> errors, no results will be saved) occurs and how to fix it? If yes,
>> then please let me know.
>>
>> Thanks and Best wishes.
>>
>> Dan
>>
>>
>>
>> On Fri, Dec 2, 2011 at 5:08 AM, Benjamin Volland<volland@econ.mpg.de>
>>  wrote:
>>>
>>> Hi Danny,
>>>
>>> your most easy option is to follow Jeph's second suggestion.
>>> That is, run your estimation once on the full sample. Then use
>>> - matrix b = e(b) - and then simply re-use the same matrix as starting
>>> values for each bootstrap iteration, using the -from(b)- option in your
>>> -xtprobit- command.
>>> You'll probably have to check whether this increases estimation speed
>>> substantially using a small number of bootstrap interations, and simply
>>> compare the time with and without setting initial values.
>>> If you want to use different starting values in each iteration (though I
>>> somehow fail to see the improvement given that each bootstrap iteration
>>> simply draws a new sample from the original data), i.e.
>>> write your own bootstrap program, check the Sata FAQ from the UCLA on how
>>> to
>>> do it:
>>> http://www.ats.ucla.edu/stat/stata/faq/ownboot.htm
>>> Again, whether that substantially increases the speed of your estimation
>>> you
>>> would have to check.
>>>
>>> Best, Ben
>>>
>>>
>>>
>>> On 01/12/2011 23:41, Danny Dan wrote:
>>>>
>>>>
>>>> Thank you so much for the answer Jeph. However, I do not know how to
>>>> write the code as mentioned in your reply. If you can give me further
>>>> insight that would be really helpful.
>>>>
>>>> Thanks once again.
>>>>
>>>> On Sun, Nov 27, 2011 at 10:06 AM, Jeph Herrin<stata@spandrel.net>
>>>>  wrote:
>>>>>
>>>>>
>>>>> One trick is to re-use the estimation from each model as the starting
>>>>> values
>>>>> for the next one. So after calling xtprobit, save the matrix e(b) and
>>>>> then
>>>>> use it with the -from(b)- option.
>>>>>
>>>>>   xtprobit ...
>>>>>   matrix b=e(b)
>>>>>   xtprobit ...., from(b)
>>>>>
>>>>> But to do this, I think you'll have to write your own bootstrap code
>>>>> (not
>>>>> too difficult).
>>>>> Or you may get by with using -bootstrap:-, if you reuse the same matrix
>>>>> for
>>>>> each
>>>>> bootstrap.
>>>>>
>>>>> Hope this helps,
>>>>> Jeph
>>>>>
>>>>>
>>>>>
>>>>> On 11/26/2011 10:31 PM, Danny Dan wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Dear All,
>>>>>>
>>>>>> How to increase computation speed while generating standard errors
>>>>>> using bootstrapping for panel data probit model (xtprobit,
>>>>>> vce(bootstrap))?
>>>>>>
>>>>>> Reduction in the number of replications (reps()) is of no help.
>>>>>>
>>>>>> Please let me know.
>>>>>>
>>>>>> Thank you,
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Malini
>>>>>> *
>>>>>> *   For searches and help try:
>>>>>> *   http://www.stata.com/help.cgi?search
>>>>>> *   http://www.stata.com/support/statalist/faq
>>>>>> *   http://www.ats.ucla.edu/stat/stata/
>>>>>>
>>>>>>
>>>>>> -----
>>>>>> No virus found in this message.
>>>>>> Checked by AVG - www.avg.com
>>>>>> Version: 2012.0.1873 / Virus Database: 2101/4642 - Release Date:
>>>>>> 11/27/11
>>>>>>
>>>>>>
>>>>>>
>>>>> *
>>>>> *   For searches and help try:
>>>>> *   http://www.stata.com/help.cgi?search
>>>>> *   http://www.stata.com/support/statalist/faq
>>>>> *   http://www.ats.ucla.edu/stat/stata/
>>>>
>>>>
>>>>
>>>> *
>>>> *   For searches and help try:
>>>> *   http://www.stata.com/help.cgi?search
>>>> *   http://www.stata.com/support/statalist/faq
>>>> *   http://www.ats.ucla.edu/stat/stata/
>>>>
>>>>
>>>
>>> *
>>> *   For searches and help try:
>>> *   http://www.stata.com/help.cgi?search
>>> *   http://www.stata.com/support/statalist/faq
>>> *   http://www.ats.ucla.edu/stat/stata/
>>
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index