st: reshape a large wide longitudinal data set to long

[Amanda Fu <mandy.fu1@gmail.com>]

Hi all,

I was wondering if anyone could give me some suggestion on reshaping a
large wide longitudinal data set to long version.

I am working on a wide version of longitudinal set : about 10000
observations, 2000 variables totally for all the 10 years, data set
size :  113,440,212.  It is wide because teh original  data set is
wide.  Now I would like to reshape it into a long version, since most
analysis can be done in long version. But it is not surprising  the
--reshape---command can not be done because the data set is too large.
Stata suggests me to either increase memory, or to drop variables or
observations.

The thing is, I have not finally decided what variables are going to
be used in the following analysis yet .  If I drop variables, it will
cause the inconvenience that I might have to reshape again and again
to add variables in the long version. Definitely I will not try to use
all the 2000 variables in the analysis, but it is still painful to
re-do the reshaping thing to add extra variables.

I was thinking a clumsy way: I break down the original wide data set
into several small wide data sets , reshape them separately, and then
append all the small long data sets together. Is this way OK?

Actually, what I am more interested in is , how the experts deal with
this kind of problem?

Thanks for your time!

Sincerely,
Amanda
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/