Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
Nick Cox <njcoxstata@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: Improving processing speed |

Date |
Tue, 29 May 2012 08:34:02 +0100 |

One detail is that you have lines in couples like this keep if sizef==`i' nl gom3: P_WOR ageyear if sizef==`i' The second -if sizeof==`i'- is redundant given the first. But that may not help much. If this is still running, you probably just have to be patient. But the key question is what is the slowest part? I'd guess wildly it's one or more of your -nl- calls. Nick On Tue, May 29, 2012 at 7:14 AM, Sebastian Galarza <sebastian@theicct.org> wrote: > Hi all, > > I have the following code and it is taking a very long time to loop through the dataset. In summary, I am running different regressions and retrieving a matrix with the beta values and r-squared for each regression. I am far from being an expert user and would like to know if there are any improvements that I can make to increase the processing speed as it is took over 20 hours to get through the first loop the second loop is still running after about the same amount of time. As a side note I originally had forvalues i=1/17,000 but given the processing time I decided to use a sample to obtain preliminary results. > > Any help would be greatly appreciated, > > clear all > > *START - Exponential Regression > forvalues i=1/376 { > use "/Users/Sebastian/Desktop/Matrix/Pool_Sizef_Sample.dta", clear > keep if sizef==`i' > nl (P_WOR = {b0=0.1}*(1 * exp({b1=0.05}* (ageyear)))) if sizef==`i' > matrix beta = e(b) > matrix r2=e(r2) > matrix list beta > matrix list r2 > matrix D = beta, r2 > svmat D, names(D) > keep fieldnumber poolnumber sizef D1 D2 D3 > keep if D1!=. > duplicates drop > save "/Users/Sebastian/Desktop/Matrix/Sample/exponential/pool`i'.dta", replace > clear matrix > clear > program drop _all > } > > use "/Users/Sebastian/Desktop/Matrix/Sample/exponential/pool1.dta", clear > forvalues i=2/376 { > append using "/Users/Sebastian/Desktop/Matrix/Sample/exponential/pool`i'.dta" > sort sizef > save "/Users/Sebastian/Desktop/Matrix/Sample/exponential/exponential.dta", replace > } > > clear > program drop _all > > *END - Exponential Regression > > clear all > *START - Logistical Function > forvalues i=1/376 { > use "/Users/Sebastian/Desktop/Matrix/Pool_Sizef_Sample.dta", clear > keep if sizef==`i' > nl log3: P_WOR ageyear if sizef==`i' > matrix beta = e(b) > matrix r2=e(r2) > matrix list beta > matrix list r2 > matrix D = beta, r2 > svmat D, names(D) > keep fieldnumber poolnumber sizef D1 D2 D3 D4 > keep if D1!=. > duplicates drop > save "/Users/Sebastian/Desktop/Matrix/Sample/logistical/pool`i'.dta", replace > clear matrix > clear > program drop _all > } > > use "/Users/Sebastian/Desktop/Matrix/Sample/logistical/pool1.dta", clear > forvalues i=2/376 { > append using "/Users/Sebastian/Desktop/Matrix/Sample/logistical/pool`i'.dta" > sort sizef > save "/Users/Sebastian/Desktop/Matrix/Sample/logistical/logistical.dta", replace > } > clear > program drop _all > * END - Logistical Function > > > *START - Gompertz Function > forvalues i=1/376 { > use "/Users/Sebastian/Desktop/Matrix/Pool_Sizef_Sample.dta", clear > keep if sizef==`i' > nl gom3: P_WOR ageyear if sizef==`i' > matrix beta = e(b) > matrix r2=e(r2) > matrix list beta > matrix list r2 > matrix D = beta, r2 > svmat D, names(D) > keep fieldnumber poolnumber sizef D1 D2 D3 D4 > keep if D1!=. > duplicates drop > save "/Users/Sebastian/Desktop/Matrix/Sample/gompertz/pool`i'.dta", replace > clear > clear matrix > program drop _all > } > > use "/Users/Sebastian/Desktop/Matrix/Sample/gompertz/pool1.dta", clear > forvalues i=2/376 { > append using "/Users/Sebastian/Desktop/Matrix/Sample/gompertz/pool`i'.dta" > sort sizef > save "/Users/Sebastian/Desktop/Matrix/Sample/gompertz/gompertz.dta", replace > } > clear > program drop _all > * END - Gompertz Function > > clear > program drop _all > * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**References**:**st: Improving processing speed***From:*Sebastian Galarza <sebastian@theicct.org>

- Prev by Date:
**Re: st: How to apply a command to numeric variables only** - Next by Date:
**RE: st: How to assign distribution of one variable to the other variable** - Previous by thread:
**st: Improving processing speed** - Index(es):