RE: st: performance benchmarking stata (and mata) programs and improving program performance

["Brent McSharry (ADHB)" <BrentM@adhb.govt.nz>]

Thank you very much to both Mike and Sergiy

Using the timer functions, I was able to ascertain that no optimizations would make any difference away from the nested loop (ie over 99% of the time was spent in the nested part of the loop). Nonetheless the _st_data function was used as it can't hurt. Using the link to 

http://www.stata.com/statalist/archive/2006-11/msg00808.html

gave the hint to use matrix multiplication (along with the range subscript notation (vertical bar within square brackets)[| ... |]), rather than a nested loop where possible.

Using the code below sped up execution on 6500 records from over 6.5 seconds to 250 milliseconds.

As per Sergiy erudite comments, the power calculation will approach zero as N approaches infinity. In reality, the scalar where 0 can be safely used as an approximation will be related to the resolution of the screen or print media. The program is sufficiently fast, that I have not implemented this at this stage, but worth thinking about if average run lengths were to get higher still.

Thank you all

Brent
---------------------------------------------
	for (j=1;j<=usedNobs;j++) 
	{
		currObs = ewmaObs[j,1] = _st_data(j,oIndx)*smooth + currObs *oneMinusSm
		expectJ = _st_data(j,eIndx)
		currExp = ewmaExp[j,1] = expectJ*smooth + currExp *oneMinusSm
		cumSeCalc = expectVar[j] = expectJ*(1-expectJ)
		ewmaSe[j,1] = smooth*sqrt(smoothPow[|1,(usedNobs-j+1) \ 1,usedNobs|] * expectVar[|1,1 \ j,1|])
	}

//build lookup for power calculations (which tend to be computationally expensive)
real rowvector function getSmoothPow(real scalar oneMinusSm, real scalar length)
{
	real scalar i, oneMinusSm2, currPow
	real rowvector returnVar
	
	oneMinusSm2 = oneMinusSm * oneMinusSm
	returnVar = J(1,length,.)
	currPow = returnVar[1,length] = 1
	for (i=length-1;i>0;i--) 
	{
		returnVar[1,i] = currPow = oneMinusSm2 * currPow
	}
	return(returnVar)
}


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/