Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
From | Austin Nichols <austinnichols@gmail.com> |
To | statalist@hsphsun2.harvard.edu |
Subject | Re: st: RE: Using macros and batch running. |
Date | Mon, 4 Jul 2011 08:26:00 -0400 |
Adrian Sayers <adrian.sayers@bristol.ac.uk> : Use an ado file that defines a -program- to do everything you are currently -display-ing to a log file below. The -program- creates data, does estimation etc., then reports results as scalars. Then run -simulate- to call that program. If the contents of the program are not well parallelized, then you can benefit by calling multiple instances of Stata because Stata's parallelization is small-grain, not large-grain. You need to calculate the seed to start from, though. Let's say you calculate that the -program- uses no more than 3000 random numbers (you need to check this carefully, as many programs you use may generate random numbers). Then you can run 1000 simulations using 10 instances of Stata each running 100 simulations by writing a do file that has three lines to just run -simulate- and save the results, then writing another do file that calls the ancillary do file 10 times (also incrementing the seed 3000 times, or as required). If there are outputs from one simulation that affect a later simulation, then you really do have a serial process and this will not work, but the example code you gave does not illustrate such a case. -----begin ado file myprog.ado-------- prog myprog, rclass syntax [anything] [, opt(string) ] * machinery here return scalar result=r(mean) eret clear end -----end do file-------- -----begin ancillary do file mysim.do-------- args seed id simulate, r(100) seed(`seed'): myprog x y z, opt(whatever) save run`id' -----end do file-------- --------begin main do file--------- set seed 1 forv i=1/10 { winexec "\\mypath\stata11\StataMP.exe" -b do mysim `c(seed)' `i' forv r=1/3000 { loc u=uniform() } } ---------end main do file---------- On Mon, Jul 4, 2011 at 7:17 AM, Adrian Sayers <adrian.sayers@bristol.ac.uk> wrote: > Hi Daniel, > > I had thought of righting a header file. (i.e. .doh) but i am not sure this > will solve my problem. I should probably explain what i am trying to do > better. > > Currently i am running series of simulations, which take 3+ days to complete > on my PC. > > Most of the process are serial, therefore their is limited gains in > parrelisation. However i had hoped that i could invoke stata in numerous > windows, this would allow me to perform all experiments in the simulation > simultaneously, drastically cutting the running time from 3 days to 20 > minutes. > > To do this i need to modify do files on the fly, so to speak. Typically if > you were running this on only 1 machine i would bundle this in a loop and > just wait for it to run serially. > > My second attempt is to write a do file, which in turns writes all my other > do files. > > But this is not pretty, or very elegant. > Hopefully this will give you a better idea of what i am trying to achieve. > > Cheers > Adrian > > E.g. > **************************************************************************** > ** > * Create the do files in code... Basically Prefixing all code with display > foreach i in 1 2 3 { > cap log close > qui log using test`i'.do , text replace > di "clear all " > di "set obs 1000 " > di "gen id = _n " > di "gen x = `i'*uniform() " > di "save testdata`i' , replace " > > > qui log close > } > > * execute batch created do files using the batch function. > forvalues i = 1/3 { > winexec "\\mypath\stata11\StataMP.exe" -e include test`i'.do > } > > **************************************************************************** * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/