[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

From |
"Sergiy Radyakin" <serjradyakin@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: RE: st: Memory Issues |

Date |
Fri, 4 Apr 2008 16:25:14 -0400 |

1. Martin, perhaps you could tell us something more specific about your data. 2. The error message you quote usually appears after set mem command. AFAIK graphing commands do not set memory. The only possibility I see, is that the graphing subsystem could not allocate resources for creating a new window, and graphical objects. Don't forget, that each point on a scatter will be a new object in windows, with it's own pointer (4 bytes), color(4 bytes), size (N bytes), type (M bytes), etc. So if you are doing a scatter with 800Million observations, you can be sure to run out of resources. By increasing memory given to Stata you are making things WORSE. Stata is allocated it's 2GB by Windows automatically when it is launched. What you adjust by -set mem- is the amount of memory available for storing the DATA, and NOT for the other objects (graphs, matrices, anything you see on the screen, including command line, output window, etc). To see this: set mem 100k sysuse auto expand 26 memory // you have ~4kb free matrix A=J(100,100,1.23456789) // matrix A requires 100*100*8 bytes ~ 80kb // no error: matrix list A This is because matrix A was stored into the RESIDUAL memory of 2GB minus whatever Stata code requires minus 100kb. Usually we do not care, since the dataset is our most "heavy" object. But things become more complicated, when you start working with other "heavy" objects, like plugins, which require memory for their own work, and in my case that can be 600M and more. Then you start carefully considering whether you need to give Stata that much memory (see my previous inquiries regarding how to estimate the amount needed for a particular dataset). 3. Finally, there is a concept of "serset" in Stata. When you save a graph, all data required to build the graph will be saved with it to disk. (to see this, just save your graph as .gph and browse the file with any hex viewer . You will find <BeginSersetData> and <EndSersetData> with binary values in between ). This allows Stata to replay the graph later. These sersets must also be residing somewhere, so just having enough place to store your data, does not necessarily mean you can plot the data. In the worst case, you might need twice as much memory and good balancing of memory between different allocations. Here "twice" is optimistic, since Stata (I suspect) also remembers the sequence of graphical commands needed to repaint the graph a la Metafile, which can change twice to tripple (in the worst case). The worst case (as I see it) is a very long narrow dataset, sort of x-y pairs for a scatter, that occupy hundreds of megabytes (see #1). 4. From what I know, Vista should do better then XP in giving Stata more memory. A typical limit for Stata 10 would be about 700-800 MB on XP 32-bit, and about 1500MB on Vista 32-bit. Best regards, Sergiy Radyakin On 4/3/08, Martin Weiss <martin.weiss@uni-tuebingen.de> wrote: > If Stata refuses to draw a simple graph at that level of memory, you bet I > do... > > Martin Weiss > _________________________________________________________________ > > Diplom-Kaufmann Martin Weiss > Mohlstrasse 36 > Room 415 > 72074 Tuebingen > Germany > > Fon: 0049-7071-2978184 > > Home: http://www.wiwi.uni-tuebingen.de/cms/index.php?id=1130 > > Publications: http://www.wiwi.uni-tuebingen.de/cms/index.php?id=1131 > > SSRN: http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=669945 > > > -----Original Message----- > From: owner-statalist@hsphsun2.harvard.edu > [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Tamas Bartus > (tbartus) > Sent: Thursday, April 03, 2008 10:45 AM > To: statalist@hsphsun2.harvard.edu > Subject: Re: RE: st: Memory Issues > > Stata people often told us that allocating too much memory is as bad as > allocating not enough memory. Do you really need 800m? > > tamas > > > ------------------------------------------------ > > Tamas Bartus, PhD > Associate Professor, Institute of Sociology and Social Policy > Corvinus University, Budapest > 1093 Budapest, Közraktár utca 4-6. > Phone: +36-1-482-7301 > Fax: +36-1-482-7348 > Homepage: www.uni-corvinus.hu/bartus > > > ----- Eredeti üzenet ----- > Feladó: Martin Weiss <martin.weiss@uni-tuebingen.de> > Dátum: Csütörtök, Április 3, 2008 10:41 de > Tárgy: RE: st: Memory Issues > Címzett: statalist@hsphsun2.harvard.edu > > > > Well, everything is up to date, I checked that beforehand... > > > > Martin Weiss > > _________________________________________________________________ > > > > Diplom-Kaufmann Martin Weiss > > Mohlstrasse 36 > > Room 415 > > 72074 Tuebingen > > Germany > > > > Fon: 0049-7071-2978184 > > > > Home: http://www.wiwi.uni-tuebingen.de/cms/index.php?id=1130 > > > > Publications: http://www.wiwi.uni-tuebingen.de/cms/index.php?id=1131 > > > > SSRN: http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=669945 > > > > -----Original Message----- > > From: owner-statalist@hsphsun2.harvard.edu > > [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Viktor > Slavtchev > > Sent: Thursday, April 03, 2008 10:26 AM > > To: statalist@hsphsun2.harvard.edu > > Subject: Re: st: Memory Issues > > > > I had the same problem. > > However, everything was fine after updating Stata 10: > > update all > > update swap > > > > HTH > > Viktor > > > > Martin Weiss wrote: > > > Dear Statalisters, > > > > > > I know this is an evergreen, but it is still annoying to have Stata > > arguing > > > that "op. sys. refuses to provide memory"... (r909). How can it possibly > > be > > > that on a machine with an AMD Athlon 3200+, 2.00 GHz and 1.5G of RAM > > > Stata/SE 10.0 does not even draw a twoway-graph with 3 components (2 > > > scatters, one lfit)? I have allocated 800m to it. When I attempt to > > increase > > > this allocation via -set mem- it gives me a warning "memory reallocation > > > failed" and exits without further ado. All unsaved work is lost in this > > > process. > > > > > > I am considering several hypotheses as to the likely cause, one > > being the > > > Vista SP1 I installed yesterday, but maybe the list has other ideas > > how I > > > can make this work... > > > > > > > > > Warm regards, > > > Martin Weiss > > > > > > > > > > > > * > > > * For searches and help try: > > > * http://www.stata.com/support/faqs/res/findit.html > > > * http://www.stata.com/support/statalist/faq > > > * http://www.ats.ucla.edu/stat/stata/ > > > > > * > > * For searches and help try: > > * http://www.stata.com/support/faqs/res/findit.html > > * http://www.stata.com/support/statalist/faq > > * http://www.ats.ucla.edu/stat/stata/ > > > > * > > * For searches and help try: > > * http://www.stata.com/support/faqs/res/findit.html > > * http://www.stata.com/support/statalist/faq > > * http://www.ats.ucla.edu/stat/stata/ > > * > * For searches and help try: > * http://www.stata.com/support/faqs/res/findit.html > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > > > * > * For searches and help try: > * http://www.stata.com/support/faqs/res/findit.html > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > * * For searches and help try: * http://www.stata.com/support/faqs/res/findit.html * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**Re: RE: st: Memory Issues***From:*"Gabi Huiber" <ghuiber@gmail.com>

**st: Margeff & bivariate probit***From:*Martin Watts <Martin.Watts@newcastle.edu.au>

**References**:**Re: st: Memory Issues***From:*Viktor Slavtchev <viktor.slavtchev@uni-jena.de>

**Re: RE: st: Memory Issues***From:*"Tamas Bartus (tbartus)" <tamas.bartus@uni-corvinus.hu>

- Prev by Date:
**Re: st: Stata graph how to?** - Next by Date:
**st: RE: RE: how to simulate an arbitrary distribution** - Previous by thread:
**RE: st: exporting results** - Next by thread:
**st: Margeff & bivariate probit** - Index(es):

© Copyright 1996–2015 StataCorp LP | Terms of use | Privacy | Contact us | What's new | Site index |