Re: st: RE: Large data set that won't open

[Kay Walker <kay.walker@internode.on.net>]

Possibly if you can't get into Stata to edit the data file to make it 
smaller, drop it into Excel, split the file on variables not cases,(and 
copy the identifiers to the "new" half)  so that you can analyse only 
those variables and cases you NEED in a particular context. You can 
always swap relevant variables in and out of the working data file if 
you want to perform further analyses later. I think it's the fact that 
the RAM has to hold the program and the relevant part of the datafile in 
memory for analysis that is causing the problem, not merely the size of 
the datafile vs the program.

Kay (Adelaide)

Martin Weiss wrote:
> <>
>
> The size of your RAM is hard to tell from your current memory allocation -
> the latter is just a lower bound for the former. But I guess you do not have
> 6G of RAM, do you? You would still need additional RAM for computations,
> then. 
>
> Note you can -use- a dataset with -if- and -in- qualifiers, so it is still
> possible to get a subset into mem...
>
>
> HTH
> Martin
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of John Antonakis
> Sent: Donnerstag, 24. Juni 2010 09:37
> To: statalist@hsphsun2.harvard.edu
> Subject: st: Large data set that won't open
>
> Hi:
>
> I have a rather large data set that I cannot open in Stata; here are the 
> particulars:
>
> ls "D:\My Documents\STATA\upstata.dta",
> 5375.8M   6/23/10 10:44  upstata.dta  
>
> When I try to open the data set I get:
>
> . use "D:\My Documents\STATA\upstata.dta", clear
> no room to add more observations
>     An attempt was made to increase the number of observations beyond 
> what is currently
>     possible.  You have the following alternatives:
>
>      1.  Store your variables more efficiently; see help compress.  
> (Think of Stata's data
>          area as the area of a rectangle; Stata can trade off width and 
> length.)
>
>      2.  Drop some variables or observations; see help drop.
>
>      3.  Increase the amount of memory allocated to the data area using 
> the set memory
>          command; see help memory.
> r(901);
>
> Here is my current memory allocation:
>
> . q memory
>
> Current memory allocation
>
>                     current                                 memory usage
>     settable          value     description                 (1M = 1024k)
>     --------------------------------------------------------------------
>     set maxvar         5000     max. variables allowed           1.909M
>     set memory         1024M    max. data space              1,024.000M
>     set matsize        1000     max. RHS vars in models          7.713M
>                                                             -----------
>                                                              1,033.622M
>
>
> It seems that I just don't have enough RAM to open this file.  Is that 
> correct?
>
> Best,
> J.
>
>   

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/