Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: How many observations can Stata 11 handle?


From   Richard Williams <richardwilliams.ndu@gmail.com>
To   statalist@hsphsun2.harvard.edu, statalist@hsphsun2.harvard.edu
Subject   Re: st: How many observations can Stata 11 handle?
Date   Wed, 15 Feb 2012 09:22:06 -0500

At 09:02 AM 2/15/2012, Maarten Buis wrote:
On Wed, Feb 15, 2012 at 2:42 PM, Tunga Kantarcı wrote:
> Here is a short question. What is the maximum number of observations
> that Stata 11 can handle?

-help limits-

> I have an administrative data set which
> includes observations for about 3 million unique individuals, for 20
> variables.

I very much doubt that the maximum number of observations is the
problem here (unless you are using Small Stata, in which case you get
what you pay for). Instead I suspect a problem with memory,
specifically I suspect that one or more variables are stored as very
long strings. The solution to that is to either not load that/those
variable(s) by using -use varlist using filename- instead of -use
filename-. You can spot the offending variables using -desc using
filename-. Alternatively you can load only the offending variables,
fix them and merge them with the rest of the variables.

> I plan to draw a sample from the data set using SPSS and transfer the
> data to Stata 11 with the Stat-transfer data conversion program.

no need to use SPSS: to draw a 70% sample before opening the data you
can type -use <filename> if runiform() < .7-

Stat transfer can also draw a random sample for you. This may be better if the size of the data set is a problem for Stata. See the Observations tab in Stat Transfer. Also see the Options tab for setting the seed if you want to be able to reproduce the exact same sample later. Stat Transfer also tells you how the vars are stored, which can be helpful for identifying string vars if that is what is causing the problem.


-------------------------------------------
Richard Williams, Notre Dame Dept of Sociology
OFFICE: (574)631-6668, (574)631-6463
HOME:   (574)289-5227
EMAIL:  Richard.A.Williams.5@ND.Edu
WWW:    http://www.nd.edu/~rwilliam


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index