Statalist The Stata Listserver


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

RE: st: file size problems


From   "Scott Merryman" <smerryman@kc.rr.com>
To   <statalist@hsphsun2.harvard.edu>
Subject   RE: st: file size problems
Date   Sun, 20 Aug 2006 11:31:48 -0500

Regarding the third question:  rather than subset the data after it is in
memory try loading only the relevant subset, 

 -use first.dta if month<mdy(1,1,2000)-

For example:

. set memory 1m

Current memory allocation

                    current                                 memory usage
    settable          value     description                 (1M = 1024k)
    --------------------------------------------------------------------
    set maxvar         5000     max. variables allowed           1.733M
    set memory            1M    max. data space                  1.000M
    set matsize         400     max. RHS vars in models          1.254M
                                                            -----------
                                                                 3.987M

. sysuse auto
(1978 Automobile Data)

. expand 275
(20276 observations created)

. keep if fore ==1
no room to add more variables due to width
    An attempt was made to add a variable that would have increased the
memory required to store an observation beyond what is currently possible.
You have the following alternatives:

     1.  Store existing variables more efficiently; see help compress.

     2.  Drop some variables or observations; see help drop.  (Think of
Stata's data area as the area of a rectangle; Stata can trade off width and
         length.)

     3.  Increase the amount of memory allocated to the data area using the
set memory command; see help memory.
r(902);

. save "C:\Documents and Settings\Scott_2\Desktop\temp.dta", replace
file C:\Documents and Settings\Scott_2\Desktop\temp.dta saved

. clear

. use "C:\Documents and Settings\Scott_2\Desktop\temp.dta" if fore == 1
(1978 Automobile Data)


Scott


> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> statalist@hsphsun2.harvard.edu] On Behalf Of Jeph Herrin
> Sent: Sunday, August 20, 2006 10:50 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: file size problems
> 
> Sorry, perhaps I should have been more clear. I have 3 distinct questions:
> 
> 1) Why does a 500mb file require 800mb of Stata memory?
> 2) How to get around "op. sys. refuses to supply memory" error
>     when using -set mem 1000m- in winxp (when xp should be able
>     to access at least 2.5gb, and no other apps are running)?
> 3) How to drop observations once the file is in memory, when
>     -keep if mycondition- gives a "no room to add more obs" error
>     (and there is no way to increase memory)?
> 
> thanks,
> Jeph
> 
> 


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index