Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: Re: Re: RE: Time taken for reshape command


From   "Michael Blasnik" <michael.blasnik@verizon.net>
To   <statalist@hsphsun2.harvard.edu>
Subject   st: Re: Re: RE: Time taken for reshape command
Date   Tue, 18 May 2004 12:33:15 -0400

I would allocate much more ram to Stata, maybe 200m -- your dataset has
about 1,000,000 values and reshape commands can take a lot more memory than
the size of the dataset that is being reshaped because of how results are
built up.  Still, large reshapes like this can still take longer than you
might think.  If the increased memory doesn't result in fairly quick
reshapes, you could try doing it in chunks -- split up the original file
into maybe 5 chunks (about 200 obs each) and reshape each of those and then
combine the results.  This approach can improve speed noticeably (and can be
done fairly easily in a do file), at least in large datasets.

Michael Blasnik
michael.blasnik@verizon.net


----- Original Message ----- 
From: "Subhankar Nayak" <nayak@bellsouth.net>
To: <statalist@hsphsun2.harvard.edu>
Sent: Tuesday, May 18, 2004 12:13 PM
Subject: st: Re: RE: Time taken for reshape command


> Sorry for missing out those details.
> My machine is 2.33GHz, 1GB RAM
> I have allocated 32M memory to Stata
>
> Thanks.
>
> ----- Original Message ----- 
> From: "Nick Cox" <n.j.cox@durham.ac.uk>
> To: <statalist@hsphsun2.harvard.edu>
> Sent: Tuesday, May 18, 2004 12:07 PM
> Subject: st: RE: Time taken for reshape command
>
>
> > I would have thought that more information
> > was needed on your machine set-up and how
> > much memory you asked of Stata to assess this.
> >
> > Nick
> > n.j.cox@durham.ac.uk
> >
> > > -----Original Message-----
> > > From: owner-statalist@hsphsun2.harvard.edu
> > > [mailto:owner-statalist@hsphsun2.harvard.edu]On Behalf Of Subhankar
> > > Nayak
> > > Sent: 18 May 2004 17:05
> > > To: statalist@hsphsun2.harvard.edu
> > > Subject: st: Time taken for reshape command
> > >
> > >
> > > Hello all,
> > >
> > > The "reshape" command that I am using takes an extremely long
> > > time and I am
> > > not sure whether this is normal.
> > > Let me mention the details.
> > >
> > > I have 1000 observations and 926 variables.
> > >
> > > Variable: "ID"  takes values from 1 through 1000. (denoting
> > > 1000 companies)
> > > Variable  "Y"   is a specific corporate value
> > > 924 variables: X1 through X924 denote the 924 past stock prices.
> > >
> > > I am reshaping it in the long format into 924000 observations and 3
> > > variables (ID,Y,X) using
> > > reshape long X, i(ID) j(DAYS)
> > >
> > > This single command takes 10-12 minutes to process.
> > > Is this normal?
> > >
> > > Thanks.


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index