st: Improving processing speed

[Sebastian Galarza <sebastian@theicct.org>]

Hi all, 

I have the following code and it is taking a very long time to loop through the dataset. In summary, I am running different regressions and retrieving a matrix with the beta values and r-squared for each regression. I am far from being an expert user and would like to know if there are any improvements that I can make to increase the processing speed as it is took over 20 hours to get through the first loop the second loop is still running after about the same amount of time. As a side note I originally had forvalues i=1/17,000 but given the processing time I decided to use a sample to obtain preliminary results. 

Any help would be greatly appreciated,

clear all

*START - Exponential Regression
forvalues i=1/376 {
use "/Users/Sebastian/Desktop/Matrix/Pool_Sizef_Sample.dta", clear
keep if sizef==`i'
nl (P_WOR = {b0=0.1}*(1 * exp({b1=0.05}* (ageyear)))) if sizef==`i'
matrix beta = e(b)
matrix r2=e(r2)
matrix list beta 
matrix list r2   
matrix D = beta, r2
svmat D, names(D) 
keep fieldnumber poolnumber sizef D1 D2 D3
keep if D1!=.
duplicates drop
save "/Users/Sebastian/Desktop/Matrix/Sample/exponential/pool`i'.dta", replace
clear matrix
clear
program drop _all
}

use "/Users/Sebastian/Desktop/Matrix/Sample/exponential/pool1.dta", clear	
forvalues i=2/376 {
append using "/Users/Sebastian/Desktop/Matrix/Sample/exponential/pool`i'.dta"
sort sizef
save "/Users/Sebastian/Desktop/Matrix/Sample/exponential/exponential.dta", replace
}

clear
program drop _all

*END - Exponential Regression

clear all
*START - Logistical Function
forvalues i=1/376 {
use "/Users/Sebastian/Desktop/Matrix/Pool_Sizef_Sample.dta", clear
keep if sizef==`i'
nl log3: P_WOR ageyear if sizef==`i'
matrix beta = e(b)
matrix r2=e(r2)
matrix list beta 
matrix list r2   
matrix D = beta, r2
svmat D, names(D) 
keep fieldnumber poolnumber sizef D1 D2 D3 D4
keep if D1!=.
duplicates drop
save "/Users/Sebastian/Desktop/Matrix/Sample/logistical/pool`i'.dta", replace
clear matrix
clear
program drop _all
}

use "/Users/Sebastian/Desktop/Matrix/Sample/logistical/pool1.dta", clear	
forvalues i=2/376 {
append using "/Users/Sebastian/Desktop/Matrix/Sample/logistical/pool`i'.dta"
sort sizef
save "/Users/Sebastian/Desktop/Matrix/Sample/logistical/logistical.dta", replace
}
clear
program drop _all
* END - Logistical Function


*START - Gompertz Function
forvalues i=1/376 {
use "/Users/Sebastian/Desktop/Matrix/Pool_Sizef_Sample.dta", clear
keep if sizef==`i'
nl gom3: P_WOR ageyear if sizef==`i'
matrix beta = e(b)
matrix r2=e(r2)
matrix list beta 
matrix list r2   
matrix D = beta, r2
svmat D, names(D) 
keep fieldnumber poolnumber sizef D1 D2 D3 D4
keep if D1!=.
duplicates drop
save "/Users/Sebastian/Desktop/Matrix/Sample/gompertz/pool`i'.dta", replace
clear
clear matrix
program drop _all
}

use "/Users/Sebastian/Desktop/Matrix/Sample/gompertz/pool1.dta", clear
forvalues i=2/376 {
append using "/Users/Sebastian/Desktop/Matrix/Sample/gompertz/pool`i'.dta"
sort sizef
save "/Users/Sebastian/Desktop/Matrix/Sample/gompertz/gompertz.dta", replace
}
clear
program drop _all
* END - Gompertz Function

clear
program drop _all


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/