st: Re: How to merge individual records to groups in a large dataset,w/o using collapse

["Michael Blasnik" <michael.blasnik@verizon.net>]

If you are looking at -collapse-, then  -merge- is the wrong term for Stata 
users (merge means joining tables in Stata).

You would have better luck with answers if you showed us some sample command 
and better described what end result you want.  Depending on what group 
summary statistics you want, you may be better off using just pieces of the 
data set and collapsing each one within a loop.  With very large datasets, I 
find it usually makes more sense to work from first principles in Stata and 
avoid commands like -scollapse- and most -egen- commands as well.  You can 
usually accomplish what you want more efficiently and with less memory 
overhead nad enhanced speed doing it this way.

Michael Blasnik

----- Original Message ----- 
From: "Allon Crazy" <allon_crazy@yahoo.com>
To: <statalist@hsphsun2.harvard.edu>
Sent: Monday, February 26, 2007 6:04 PM
Subject: st: How to merge individual records to groups in a large dataset, 
w/o using collapse



> I am wondering how to merge individual records to
> groups for an extremely large dataset (20 million
> observations), without using collapse. I tred
> collapse, but my computer would not offer enough
> memeory for it because the dataset is too huge. I
> tried egen, but egen does not take sampling weights
> into consideration. I am wondering whether there is
> another way or other options.
>
> I would grealy appreciate.
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/