Statalist The Stata Listserver

[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: file size problems

From   Jeph Herrin <>
Subject   Re: st: file size problems
Date   Sun, 20 Aug 2006 12:39:13 -0400

This is very helpful - guess I've been -use-ing data for so many years,
it didn't occur to me to check -help use-.  My oversight.


Scott Merryman wrote:
Regarding the third question: rather than subset the data after it is in
memory try loading only the relevant subset,
-use first.dta if month<mdy(1,1,2000)-

For example:

. set memory 1m

Current memory allocation

current memory usage
settable value description (1M = 1024k)
set maxvar 5000 max. variables allowed 1.733M
set memory 1M max. data space 1.000M
set matsize 400 max. RHS vars in models 1.254M

. sysuse auto
(1978 Automobile Data)

. expand 275
(20276 observations created)

. keep if fore ==1
no room to add more variables due to width
An attempt was made to add a variable that would have increased the
memory required to store an observation beyond what is currently possible.
You have the following alternatives:

1. Store existing variables more efficiently; see help compress.

2. Drop some variables or observations; see help drop. (Think of
Stata's data area as the area of a rectangle; Stata can trade off width and

3. Increase the amount of memory allocated to the data area using the
set memory command; see help memory.

. save "C:\Documents and Settings\Scott_2\Desktop\temp.dta", replace
file C:\Documents and Settings\Scott_2\Desktop\temp.dta saved

. clear

. use "C:\Documents and Settings\Scott_2\Desktop\temp.dta" if fore == 1
(1978 Automobile Data)


-----Original Message-----
From: [mailto:owner-] On Behalf Of Jeph Herrin
Sent: Sunday, August 20, 2006 10:50 AM
Subject: Re: st: file size problems

Sorry, perhaps I should have been more clear. I have 3 distinct questions:

1) Why does a 500mb file require 800mb of Stata memory?
2) How to get around "op. sys. refuses to supply memory" error
    when using -set mem 1000m- in winxp (when xp should be able
    to access at least 2.5gb, and no other apps are running)?
3) How to drop observations once the file is in memory, when
    -keep if mycondition- gives a "no room to add more obs" error
    (and there is no way to increase memory)?


*   For searches and help try:

*   For searches and help try:

© Copyright 1996–2015 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index