Statalist The Stata Listserver


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

RE: st: file size problems


From   "Nuno Soares" <[email protected]>
To   <[email protected]>
Subject   RE: st: file size problems
Date   Sun, 20 Aug 2006 17:02:33 +0100

Try reading the link http://www.stata.com/support/faqs/win/winmemory.html
for eventual reasons for this behaviour... However, I have a system with
1.5gb, winxp, and Stata mem defined to 900m and it loads a file with 680mb
with no problems at all (it has 3400000 observations and 57 vars). 

Best,


-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Jeph Herrin
Sent: 20 August 2006 16:50
To: [email protected]
Subject: Re: st: file size problems

Sorry, perhaps I should have been more clear. I have 3 distinct questions:

1) Why does a 500mb file require 800mb of Stata memory?
2) How to get around "op. sys. refuses to supply memory" error
    when using -set mem 1000m- in winxp (when xp should be able
    to access at least 2.5gb, and no other apps are running)?
3) How to drop observations once the file is in memory, when
    -keep if mycondition- gives a "no room to add more obs" error
    (and there is no way to increase memory)?

thanks,
Jeph


b. water wrote:
> how about using the set memory to increase Stata memory allocation? have
you tried this?
> 
> bw
> 
> 
> 
> 
> ----------------------------------------
>> Date: Sat, 19 Aug 2006 19:36:48 -0400
>> From: [email protected]
>> To: [email protected]
>> Subject: st: file size problems
>>
>> I'm using Stata 9.2 on WinXP (SP2), updated.
>>
>> I have a file which is 500mb, but when I load it into Stata, it 
>> requires 825mb. That doesn't seem right:
>>
>> . dir first.dta
>>   500.7M   8/19/06 18:33  first.dta
>>
>> . u first.dta, clear
>>
>> . d
>>
>> Contains data from first.dta
>>    obs:    75,006,197
>>   vars:             3                          19 Aug 2006 18:33
>>   size:   825,068,167 (12.6% of memory free)
>> ---------------------------------------------------
>>
>>
>> This is a problem, because WinXp will only allocate 900mb to Stata 
>> (anyone know how to change this? the machine has 4gb). The first 
>> thing I want to do when I load this file is:
>>
>> . keep if month<mdy(1,1,2000)
>>
>> however:
>>
>> no room to add more variables due to width
>>      An attempt was made to add a variable that would have increased 
>> the memory required to store an observation beyond
>>      what is currently possible.  You have the following alternatives:
>>
>>       1.  Store existing variables more efficiently; see help compress.
>>
>>       2.  Drop some variables or observations; see help drop.  (Think 
>> of Stata's data area as the area of a rectangle;
>>           Stata can trade off width and length.)
>>
>>       3.  Increase the amount of memory allocated to the data area 
>> using the set memory command; see help memory.
>> r(902);
>>
>>
>>
>> I can't drop any other variables (note that there are only 3). Any 
>> ideas? In particular, is there an alternative to "keep" that doesn't add
a variable?
>>
>> thanks,
>> Jeph
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/support/faqs/res/findit.html
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
> 
> _________________________________________________________________
> Try Live.com - your fast, personalized homepage with all the things you
care about in one place.
> http://www.live.com/getstarted
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> 
> 
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index