Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: Successive rounds of a simulation become slower and slower


From   "Lacy,Michael" <[email protected]>
To   "[email protected]" <[email protected]>
Subject   st: Successive rounds of a simulation become slower and slower
Date   Wed, 26 Oct 2011 16:34:52 +0000

Greetings,

Successive rounds of a simulation I'm doing become progressively slower, so much so as to make continuing impractical. I'd like to figure out why this is happening 
and cure it. The first round might take 2 minutes, then 5 minutes for the next, then 15, etc. Each round *should* take the same amount of time, since the same estimation process 
is being performed on similar data at each round.  


Each round involves 1000 repetitions of simulating data based on several parameters. Statistics estimated at each repetition are -post-ed, and the temporary 
-postfile- is then saved to a permanent file. In pseucode outline, the overall structure  of the code is something like this:

foreach p1 of ParamList1 {
  foreach p2 of ParamList2 {
    foreach p3 of ParamList3 {
       Initialize for this round
       forval r = 1/1000 {  // repetitions
         Set up for posting
         Create simulated data based on p1 p2 p3
         Run a user-written estimation process that uses several Mata programs
         Post the current estimates 
       }
       Save postfile to a permanent file
       Erase postfile
    }
  }
  

There are several parameters in each list, so that there are some 100+ rounds to be run, which should take a few hours in total.  But because of the round to round decrement
in performance, the overall simulation won't finish in 10-12 hours.  I'm running 11.2 SE, under Win XP, on a modest (2G RAM) machine.  There should not be any 
memory issues, since the data created for each round involves something like 10  variables for 10-20 observations; similarly, the postfile for each round is nothing big,
perhaps 10 variables for 1000 cases. I also seem to be experiencing the same problem running 11.2 SE on a Win 7 machine, with 8G.

Regards,

Mike Lacy
Colorado State University
Fort Collins CO 80523-1784
970.491.6721 (voice)

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index