Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: Successive rounds of a simulation become slower and slower


From   "Lacy,Michael" <Michael.Lacy@colostate.edu>
To   "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Subject   st: Successive rounds of a simulation become slower and slower
Date   Wed, 26 Oct 2011 16:34:52 +0000

Greetings,

Successive rounds of a simulation I'm doing become progressively slower, so much so as to make continuing impractical. I'd like to figure out why this is happening 
and cure it. The first round might take 2 minutes, then 5 minutes for the next, then 15, etc. Each round *should* take the same amount of time, since the same estimation process 
is being performed on similar data at each round.  


Each round involves 1000 repetitions of simulating data based on several parameters. Statistics estimated at each repetition are -post-ed, and the temporary 
-postfile- is then saved to a permanent file. In pseucode outline, the overall structure  of the code is something like this:

foreach p1 of ParamList1 {
  foreach p2 of ParamList2 {
    foreach p3 of ParamList3 {
       Initialize for this round
       forval r = 1/1000 {  // repetitions
         Set up for posting
         Create simulated data based on p1 p2 p3
         Run a user-written estimation process that uses several Mata programs
         Post the current estimates 
       }
       Save postfile to a permanent file
       Erase postfile
    }
  }
  

There are several parameters in each list, so that there are some 100+ rounds to be run, which should take a few hours in total.  But because of the round to round decrement
in performance, the overall simulation won't finish in 10-12 hours.  I'm running 11.2 SE, under Win XP, on a modest (2G RAM) machine.  There should not be any 
memory issues, since the data created for each round involves something like 10  variables for 10-20 observations; similarly, the postfile for each round is nothing big,
perhaps 10 variables for 1000 cases. I also seem to be experiencing the same problem running 11.2 SE on a Win 7 machine, with 8G.

Regards,

Mike Lacy
Colorado State University
Fort Collins CO 80523-1784
970.491.6721 (voice)

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index