Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: RE: st: increasing time burden during resampling


From   "Martin Weiss" <Martin.Weiss1@gmx.de>
To   statalist@hsphsun2.harvard.edu
Subject   Re: RE: st: increasing time burden during resampling
Date   Mon, 28 Dec 2009 18:08:29 +0100

<>

To be honest, I have no idea why the computations slow down. My sentiment regarding the memory allocation comes from the Stata FAQs. For instance, http://www.stata.com/support/faqs/win/memory1.html, says:

"Some users think that if giving Stata 40 MB is good, giving Stata 60 MB or even 80 MB must be better. That is not true. Once enough memory has been allocated to Stata to allow it to load the current dataset, with enough extra room for whatever temporary variables, programs, macros, etc., might be needed during analysis, no speed improvements will be seen by giving even more memory to Stata. In fact, you may hurt Stata’s performance by forcing the OS to use virtual memory if you allocate too much memory to Stata. "


The [GSW] manual also notes in "B.1" that you should "...not specify more memory than you actually need. Doing so does not benefit Stata; in fact, it may slow Stata down!"

HTH
Martin


-------- Original-Nachricht --------
> Datum: Mon, 28 Dec 2009 10:55:08 -0600
> Von: "Feiveson, Alan H. (JSC-SK311)" <alan.h.feiveson@nasa.gov>
> An: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
> Betreff: RE: st: increasing time burden during resampling

> Martin - I used "/m170" in the properties of the Stata 11 shortcut. Are
> you suggesting I make this smaller instead of larger, to make more room for
> computations? But why do the same computations take more and more memory?
> Something must be generated by -test- that clogs up the memory and doesn't
> get cleared.
> 
> 
> Westfall, P. H. and Young, S. S, 1993. Resampling-Based Multiple Testing.
> New York: John Wiley & Sons, Inc
> 
> Al
> 
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Martin Weiss
> Sent: Monday, December 28, 2009 10:32 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: increasing time burden during resampling
> 
> <>
> 
> "as described in Westfall & Young's text"
> 
> Is this a well-known text?
> 
> "I tried increasing the memory allocated to Stata, but that didn't seem to
> help"
> 
> How exactly did you do that? If it was done using -set mem-, it only
> increases the memory available to the data, which, if I understand correctly,
> did not increase during your iterations. The unnecessarily increased space
> for data will decrease the one available for Stata`s computations.
> 
> HTH
> Martin
> 
> -------- Original-Nachricht --------
> > Datum: Mon, 28 Dec 2009 10:18:52 -0600
> > Von: "Feiveson, Alan H. (JSC-SK311)" <alan.h.feiveson@nasa.gov>
> > An: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
> > Betreff: st: increasing time burden during resampling
> 
> > Hi - I am running Stata 11 on Windows XP and I am implementing a form of
> > multiple testing using a resampling method as described in Westfall &
> > Young's text. Basically, for each iteration the method is: 
> > 
> > 1)resample the data modified to a joint null situation (I use the Stata
> > -bs- command for this) 
> > 2) fit a model to the resampled modified data 
> > 3) do a bunch of tests using -test-  
> > 4) save the test results.
> > 
> > These operations are identical for each iteration, yet the time per
> > iteration increases roughly linearly until it becomes prohibitive to
> continue. 
> > 
> > If I stop after a given number of iterations, I must close out all Stata
> > processes before starting again, else the bogged down state still holds.
> > 
> > Previous versions of this that used the estimated coefficients and
> > standard errors without -test- didn't seem to have this problem. So I
> suspect it
> > has something to do with repeated use of the -test- command. Maybe
> something
> > needs to be reset or cleared after each iteration?
> > 
> > I tried increasing the memory allocated to Stata, but that didn't seem
> to
> > help. I would appreciate any suggestions for improving efficiency here.
> > 
> > 
> > Al Feiveson
> > 
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
> 
> -- 
> GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
> Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

-- 
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index