Statalist The Stata Listserver

[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

RE: st: file size problems

From   "Scott Merryman" <>
To   <>
Subject   RE: st: file size problems
Date   Sun, 20 Aug 2006 11:31:48 -0500

Regarding the third question:  rather than subset the data after it is in
memory try loading only the relevant subset, 

 -use first.dta if month<mdy(1,1,2000)-

For example:

. set memory 1m

Current memory allocation

                    current                                 memory usage
    settable          value     description                 (1M = 1024k)
    set maxvar         5000     max. variables allowed           1.733M
    set memory            1M    max. data space                  1.000M
    set matsize         400     max. RHS vars in models          1.254M

. sysuse auto
(1978 Automobile Data)

. expand 275
(20276 observations created)

. keep if fore ==1
no room to add more variables due to width
    An attempt was made to add a variable that would have increased the
memory required to store an observation beyond what is currently possible.
You have the following alternatives:

     1.  Store existing variables more efficiently; see help compress.

     2.  Drop some variables or observations; see help drop.  (Think of
Stata's data area as the area of a rectangle; Stata can trade off width and

     3.  Increase the amount of memory allocated to the data area using the
set memory command; see help memory.

. save "C:\Documents and Settings\Scott_2\Desktop\temp.dta", replace
file C:\Documents and Settings\Scott_2\Desktop\temp.dta saved

. clear

. use "C:\Documents and Settings\Scott_2\Desktop\temp.dta" if fore == 1
(1978 Automobile Data)


> -----Original Message-----
> From: [mailto:owner-
>] On Behalf Of Jeph Herrin
> Sent: Sunday, August 20, 2006 10:50 AM
> To:
> Subject: Re: st: file size problems
> Sorry, perhaps I should have been more clear. I have 3 distinct questions:
> 1) Why does a 500mb file require 800mb of Stata memory?
> 2) How to get around "op. sys. refuses to supply memory" error
>     when using -set mem 1000m- in winxp (when xp should be able
>     to access at least 2.5gb, and no other apps are running)?
> 3) How to drop observations once the file is in memory, when
>     -keep if mycondition- gives a "no room to add more obs" error
>     (and there is no way to increase memory)?
> thanks,
> Jeph

*   For searches and help try:

© Copyright 1996–2017 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index