Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: Memory Usage


From   [email protected]
To   [email protected]
Subject   Re: st: Memory Usage
Date   Wed, 14 Jul 2004 14:06:45 -0500

Tim,

There is a FAQ on approximating the memory requirements of a dataset at: http://www.stata.com/support/faqs/data/howbig.html

The basic formula is
"A back-of-the-envelope calculation for the size of a dataset is 

                               N*V*W + 4*N
number of megabytes  =  M  =  --------------
                                (1024)^2

where 


N  =  number observations
V  =  number of variables
W  =  average width in bytes of a variable

In approximating W, remember 


+-------------------------------------------------------------+
| Type of variable                               Width        |
|-------------------------------------------------------------|
| Integers,     -127 <= x <=           126          1         |
|            -32,767 <= x <=        32,766          2         |
|     -2,147,483,647 <= x <= 2,147,483,646          4         |
| Floats,                                                     |
|       single precision (default)                  4         |
|       double precision                            8         |
| strings                                      maximum length |
+-------------------------------------------------------------+"

One thing you could do, since you only have two years, is first difference data.  The estimated coefficients will be the same and it will cut your data set in half.

Something like:
xi: reg diff_nrtrgain diff_nschools ... i.instid, nocon


Hope this helps,
Scott


----- Original Message -----
From: "Tim R. Sass" <[email protected]>
Date: Wednesday, July 14, 2004 12:17 pm
Subject: st: Memory Usage

> I may be crazy, but I am trying to estimate a fixed effects model 
> with a 
> handful of explanatory variables plus 3,100 explicit dummies on a 
> two-year 
> panel data set containing 1.7 million observations (about 850,000 
> fixed 
> effects).  I am using a Sun workstation with 8GB of RAM.
> 

<snip>


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index