Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: Re: Re: Re: Re: RE: Time taken for reshape command


From   "R.E. De Hoyos" <redeho2@hotmail.com>
To   <statalist@hsphsun2.harvard.edu>
Subject   st: Re: Re: Re: Re: RE: Time taken for reshape command
Date   Tue, 18 May 2004 19:39:25 +0100

Subhankar,

See - help merge- or -help append-

Rafa

----- Original Message -----
From: "Subhankar Nayak" <nayak@bellsouth.net>
To: <statalist@hsphsun2.harvard.edu>
Sent: Tuesday, May 18, 2004 5:55 PM
Subject: st: Re: Re: Re: RE: Time taken for reshape command


> Thanks.
> I tried using "set memory 512M" but it doesn't help.
> Probably I should change the virtual memory.
>
> I will try to split the data set and work on parts.
> I will split the dataset into 5 chunks based on the ID, but I don't know
how
> to merge the 5 chunks back together.
> (I am relaticvely new to stata and hence still learning all
> commands/functions)
>
> Thanks.
>
> Subh Nayak
>
> ----- Original Message -----
> From: "Michael Blasnik" <michael.blasnik@verizon.net>
> To: <statalist@hsphsun2.harvard.edu>
> Sent: Tuesday, May 18, 2004 12:33 PM
> Subject: st: Re: Re: RE: Time taken for reshape command
>
>
> > I would allocate much more ram to Stata, maybe 200m -- your dataset has
> > about 1,000,000 values and reshape commands can take a lot more memory
> than
> > the size of the dataset that is being reshaped because of how results
are
> > built up.  Still, large reshapes like this can still take longer than
you
> > might think.  If the increased memory doesn't result in fairly quick
> > reshapes, you could try doing it in chunks -- split up the original file
> > into maybe 5 chunks (about 200 obs each) and reshape each of those and
> then
> > combine the results.  This approach can improve speed noticeably (and
can
> be
> > done fairly easily in a do file), at least in large datasets.
> >
> > Michael Blasnik
> > michael.blasnik@verizon.net
> >
> >
> > ----- Original Message -----
> > From: "Subhankar Nayak" <nayak@bellsouth.net>
> > To: <statalist@hsphsun2.harvard.edu>
> > Sent: Tuesday, May 18, 2004 12:13 PM
> > Subject: st: Re: RE: Time taken for reshape command
> >
> >
> > > Sorry for missing out those details.
> > > My machine is 2.33GHz, 1GB RAM
> > > I have allocated 32M memory to Stata
> > >
> > > Thanks.
> > >
> > > ----- Original Message -----
> > > From: "Nick Cox" <n.j.cox@durham.ac.uk>
> > > To: <statalist@hsphsun2.harvard.edu>
> > > Sent: Tuesday, May 18, 2004 12:07 PM
> > > Subject: st: RE: Time taken for reshape command
> > >
> > >
> > > > I would have thought that more information
> > > > was needed on your machine set-up and how
> > > > much memory you asked of Stata to assess this.
> > > >
> > > > Nick
> > > > n.j.cox@durham.ac.uk
> > > >
> > > > > -----Original Message-----
> > > > > From: owner-statalist@hsphsun2.harvard.edu
> > > > > [mailto:owner-statalist@hsphsun2.harvard.edu]On Behalf Of
Subhankar
> > > > > Nayak
> > > > > Sent: 18 May 2004 17:05
> > > > > To: statalist@hsphsun2.harvard.edu
> > > > > Subject: st: Time taken for reshape command
> > > > >
> > > > >
> > > > > Hello all,
> > > > >
> > > > > The "reshape" command that I am using takes an extremely long
> > > > > time and I am
> > > > > not sure whether this is normal.
> > > > > Let me mention the details.
> > > > >
> > > > > I have 1000 observations and 926 variables.
> > > > >
> > > > > Variable: "ID"  takes values from 1 through 1000. (denoting
> > > > > 1000 companies)
> > > > > Variable  "Y"   is a specific corporate value
> > > > > 924 variables: X1 through X924 denote the 924 past stock prices.
> > > > >
> > > > > I am reshaping it in the long format into 924000 observations and
3
> > > > > variables (ID,Y,X) using
> > > > > reshape long X, i(ID) j(DAYS)
> > > > >
> > > > > This single command takes 10-12 minutes to process.
> > > > > Is this normal?
> > > > >
> > > > > Thanks.
> >
> >
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/support/faqs/res/findit.html
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index