Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: Re: Re: Re: RE: Time taken for reshape command


From   "Subhankar Nayak" <nayak@bellsouth.net>
To   <statalist@hsphsun2.harvard.edu>
Subject   st: Re: Re: Re: RE: Time taken for reshape command
Date   Tue, 18 May 2004 12:55:36 -0400

Thanks.
I tried using "set memory 512M" but it doesn't help.
Probably I should change the virtual memory.

I will try to split the data set and work on parts.
I will split the dataset into 5 chunks based on the ID, but I don't know how
to merge the 5 chunks back together.
(I am relaticvely new to stata and hence still learning all
commands/functions)

Thanks.

Subh Nayak

----- Original Message ----- 
From: "Michael Blasnik" <michael.blasnik@verizon.net>
To: <statalist@hsphsun2.harvard.edu>
Sent: Tuesday, May 18, 2004 12:33 PM
Subject: st: Re: Re: RE: Time taken for reshape command


> I would allocate much more ram to Stata, maybe 200m -- your dataset has
> about 1,000,000 values and reshape commands can take a lot more memory
than
> the size of the dataset that is being reshaped because of how results are
> built up.  Still, large reshapes like this can still take longer than you
> might think.  If the increased memory doesn't result in fairly quick
> reshapes, you could try doing it in chunks -- split up the original file
> into maybe 5 chunks (about 200 obs each) and reshape each of those and
then
> combine the results.  This approach can improve speed noticeably (and can
be
> done fairly easily in a do file), at least in large datasets.
>
> Michael Blasnik
> michael.blasnik@verizon.net
>
>
> ----- Original Message ----- 
> From: "Subhankar Nayak" <nayak@bellsouth.net>
> To: <statalist@hsphsun2.harvard.edu>
> Sent: Tuesday, May 18, 2004 12:13 PM
> Subject: st: Re: RE: Time taken for reshape command
>
>
> > Sorry for missing out those details.
> > My machine is 2.33GHz, 1GB RAM
> > I have allocated 32M memory to Stata
> >
> > Thanks.
> >
> > ----- Original Message ----- 
> > From: "Nick Cox" <n.j.cox@durham.ac.uk>
> > To: <statalist@hsphsun2.harvard.edu>
> > Sent: Tuesday, May 18, 2004 12:07 PM
> > Subject: st: RE: Time taken for reshape command
> >
> >
> > > I would have thought that more information
> > > was needed on your machine set-up and how
> > > much memory you asked of Stata to assess this.
> > >
> > > Nick
> > > n.j.cox@durham.ac.uk
> > >
> > > > -----Original Message-----
> > > > From: owner-statalist@hsphsun2.harvard.edu
> > > > [mailto:owner-statalist@hsphsun2.harvard.edu]On Behalf Of Subhankar
> > > > Nayak
> > > > Sent: 18 May 2004 17:05
> > > > To: statalist@hsphsun2.harvard.edu
> > > > Subject: st: Time taken for reshape command
> > > >
> > > >
> > > > Hello all,
> > > >
> > > > The "reshape" command that I am using takes an extremely long
> > > > time and I am
> > > > not sure whether this is normal.
> > > > Let me mention the details.
> > > >
> > > > I have 1000 observations and 926 variables.
> > > >
> > > > Variable: "ID"  takes values from 1 through 1000. (denoting
> > > > 1000 companies)
> > > > Variable  "Y"   is a specific corporate value
> > > > 924 variables: X1 through X924 denote the 924 past stock prices.
> > > >
> > > > I am reshaping it in the long format into 924000 observations and 3
> > > > variables (ID,Y,X) using
> > > > reshape long X, i(ID) j(DAYS)
> > > >
> > > > This single command takes 10-12 minutes to process.
> > > > Is this normal?
> > > >
> > > > Thanks.
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index