[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]
Re: st: Re: How to merge individual records to groups in a large dataset, w/o using collapse
thanks for your advice. Seperate the dataset into
smaller ones might be better,but I did try, still too
big for my 1G ram, using "collapse".
An example I can give for what I want to accomplish is
collapse (mean) var1 var2 (median) var3 (sum) var4
I am wondering whether there is any alternative ways
to do the same task but require less ram.
--- Michael Blasnik <firstname.lastname@example.org>
> If you are looking at -collapse-, then -merge- is
> the wrong term for Stata
> users (merge means joining tables in Stata).
> You would have better luck with answers if you
> showed us some sample command
> and better described what end result you want.
> Depending on what group
> summary statistics you want, you may be better off
> using just pieces of the
> data set and collapsing each one within a loop.
> With very large datasets, I
> find it usually makes more sense to work from first
> principles in Stata and
> avoid commands like -scollapse- and most -egen-
> commands as well. You can
> usually accomplish what you want more efficiently
> and with less memory
> overhead nad enhanced speed doing it this way.
> Michael Blasnik
> ----- Original Message -----
> From: "Allon Crazy" <email@example.com>
> To: <firstname.lastname@example.org>
> Sent: Monday, February 26, 2007 6:04 PM
> Subject: st: How to merge individual records to
> groups in a large dataset,
> w/o using collapse
> >I am wondering how to merge individual records to
> > groups for an extremely large dataset (20 million
> > observations), without using collapse. I tred
> > collapse, but my computer would not offer enough
> > memeory for it because the dataset is too huge. I
> > tried egen, but egen does not take sampling
> > into consideration. I am wondering whether there
> > another way or other options.
> > I would grealy appreciate.
> * For searches and help try:
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
Don't pick lemons.
See all the new 2007 cars at Yahoo! Autos.
* For searches and help try: