Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: RE: collapse is too memory demanding

From   "Nick Cox" <>
To   <>
Subject   st: RE: collapse is too memory demanding
Date   Mon, 26 Jul 2010 20:06:58 +0100

I am not clear what help you seek. 

Stata is already indicating some things you may do. 

Note that -tabstat- is one way to display the results of these


Oliver Jones

How much additional free memory do I need to perfom a -collapse-?

Problem setting:
I have a dataset containing information about ~20 million people living
in ~180 different 
regions and working in ~330 different jobs. The information is given by
~70 zero/one dummy 
variables, like male[yes/no], female[yes/no], age20-25[yes/no], ...
When I try to collapse it like this I get the error that I need more
free memory

* begin excerpt code
* m_total is a dummy variable taking the value 1 if the person is male
* f_total is a dummy variable taking the value 1 if the person is female
collapse (sum) m_total f_total ...(68 more dummy variables), by(aoaa
beruford) fast
no room to add more variables because of width
     An attempt was made to add a variable that would have increased the
memory required to 
store an observation beyond what is currently possible.  You have the
     following alternatives:

      1.  Store existing variables more efficiently; see help compress.

      2.  Drop some variables or observations; see help drop.  (Think of
Stata's data area as 
the area of a rectangle; Stata can trade off width and length.)

      3.  Increase the amount of memory allocated to the data area using
the set memory 
command; see help memory.

.         memory
Details of set memory usage
     overhead (pointers)                     159,493,976        8.45%
     data                                  1,455,382,531       77.11%
     data + overhead                       1,614,876,507       85.56%
     free                                    272,560,293       14.44%
     Total allocated                       1,887,436,800      100.00%
Other memory usage
     set maxvar usage                          2,041,738
     set matsize usage                         1,315,200
     programs, saved results, etc.                37,424
     Total                                     3,394,362
Grand total                               1,890,831,162

* end code excerpt

*   For searches and help try:

© Copyright 1996–2015 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index