Re: st: RE: Large data set that won't open

[John Antonakis <john.antonakis@unil.ch>]

Thanks Nick; in fact, I am using Windows 7.

Best regards,
John Antonakis


On 24 juin 2010, at 10:46, "Nick Cox" <n.j.cox@durham.ac.uk> wrote:

> Also to the point: 
> 
> 1. If the dataset is too big for Stata, it will hardly fit into Excel
> (unless very recent versions of Excel have suddenly acquired greatly
> increased capacities). 
> 
> 2. I think that John signalled indirectly through an example using an
> -ls- that he is using some flavour of Unix. In that case, he is unlikely
> to have access to Excel any way. (Either way, everyone being clear about
> what OS they are using assists communication in this area.) 
> 
> Nick 
> n.j.cox@durham.ac.uk 
> 
> Martin Weiss
> 
> But why bother to use spreadsheets here, given that Stata can, as I
> mentioned earlier, -use- part of a file? Apart from -if- and -in-, a
> -varlist- can also be supplied, so the "split on variables" can be
> achieved
> within Stata.
> 
> Kay Walker
> 
> Possibly if you can't get into Stata to edit the data file to make it 
> smaller, drop it into Excel, split the file on variables not cases,(and 
> copy the identifiers to the "new" half)  so that you can analyse only 
> those variables and cases you NEED in a particular context. You can 
> always swap relevant variables in and out of the working data file if 
> you want to perform further analyses later. I think it's the fact that 
> the RAM has to hold the program and the relevant part of the datafile in
> 
> memory for analysis that is causing the problem, not merely the size of 
> the datafile vs the program.
> 
> Martin Weiss wrote:
> 
>> The size of your RAM is hard to tell from your current memory
> allocation -
>> the latter is just a lower bound for the former. But I guess you do
> not
> have
>> 6G of RAM, do you? You would still need additional RAM for
> computations,
>> then. 
>> 
>> Note you can -use- a dataset with -if- and -in- qualifiers, so it is
> still
>> possible to get a subset into mem...
> 
> John Antonakis
> 
>> I have a rather large data set that I cannot open in Stata; here are
> the 
>> particulars:
>> 
>> ls "D:\My Documents\STATA\upstata.dta",
>> 5375.8M   6/23/10 10:44  upstata.dta  
>> 
>> When I try to open the data set I get:
>> 
>> . use "D:\My Documents\STATA\upstata.dta", clear
>> no room to add more observations
>>    An attempt was made to increase the number of observations beyond 
>> what is currently
>>    possible.  You have the following alternatives:
>> 
>>     1.  Store your variables more efficiently; see help compress.  
>> (Think of Stata's data
>>         area as the area of a rectangle; Stata can trade off width
> and 
>> length.)
>> 
>>     2.  Drop some variables or observations; see help drop.
>> 
>>     3.  Increase the amount of memory allocated to the data area
> using 
>> the set memory
>>         command; see help memory.
>> r(901);
>> 
>> Here is my current memory allocation:
>> 
>> . q memory
>> 
>> Current memory allocation
>> 
>>                    current                                 memory
> usage
>>    settable          value     description                 (1M =
> 1024k)
>> 
> --------------------------------------------------------------------
>>    set maxvar         5000     max. variables allowed
> 1.909M
>>    set memory         1024M    max. data space
> 1,024.000M
>>    set matsize        1000     max. RHS vars in models
> 7.713M
>> 
> -----------
>> 
> 1,033.622M
>> 
>> 
>> It seems that I just don't have enough RAM to open this file.  Is that
> 
>> correct?
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/