Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down at the end of May, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
Austin Nichols <austinnichols@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: how can i make my loop run faster? |

Date |
Tue, 20 Sep 2011 09:49:57 -0400 |

Stefano Rossi <sr525@cornell.edu>: You can sort your data and use -in- instead of -if- to reduce processing time considerably. You can generate regression coefs and predictions without ever firing up -regress-, probably using lag operators or a wide date structure, which can also speed things up. See e.g. http://www.stata.com/statalist/archive/2008-10/msg00136.html and for some related material: http://www.stata.com/statalist/archive/2011-01/msg00662.html http://www.stata.com/statalist/archive/2008-04/msg00967.html On Tue, Sep 20, 2011 at 1:24 AM, Partho Sarkar <partho.ss+lists@gmail.com> wrote: > Hello Stefanlo > > I am not quite sure I understand what you are proposing, but I suppose > this ought to be do-able, though I can't offhand think of any direct > way to do this. I think this would also be an extremely cumbersome > way to do this, and almost certainly even slower than any of the > others we have talked about. However, I had thought of another > possible approach: > > Since you have a "short" panel- "only" 200 periods, but many more > firms, I would think dividing up your sample period into, say, 10 > (non-overlapping) subsets, and doing a statsby regression for each > would give you the results for all (or selected subset of) firms > within each time sub-period. You would still have to > combine(merge/append) the results. > > But why not try the rolling loop as suggested in the thread I cited, > first splitting the firms into manageable subsets (which I think you > would have to do in any case, if you want to run this routine- unless > you have access to a super-computer!) ? > > By the way, I am curious to know what exactly the source of your data is! > > Hope this helps > > Partho > > > On Tue, Sep 20, 2011 at 10:20 AM, Stefano Rossi <sr525@cornell.edu> wrote: >> Dear Partho, >> >> many thanks for this, which is very useful. I can see how "rolling" works, and I can see how it can generate efficiency gains, but I agree the whole procedure may still be quite slow and require splitting the sample into subsets to get a faster procedure in some way. >> >> I am currently considering a different path, namely generating a cross-section of observations by firm-period, whereby each firm-period unit contains 12 observations, from -1 to -12 (I would also have a separate data by +1 to +12). This procedure would effectively produce a dataset 12 times larger than my current one. This procedure would get around the "rolling" issue, and would allow me to use the "statsby" (or equivalent) command without worrying of the length of the estimation sample, with potentially large efficiency improvements (i.e., no "ifs"). >> >> Provided my intuition is correct, my one concern here is how to create such dataset, which is 12 times bigger than the current one. Is there a built-in Stata command that allows to do this efficiently? >> >> Many thanks for your support. >> >> Kind regards, >> >> Stefano >> >> >> ________________________________________ >> From: owner-statalist@hsphsun2.harvard.edu [owner-statalist@hsphsun2.harvard.edu] On Behalf Of Partho Sarkar [partho.ss+lists@gmail.com] >> Sent: Tuesday, September 20, 2011 12:33 AM >> To: statalist@hsphsun2.harvard.edu >> Subject: Re: st: how can i make my loop run faster? >> >> I guess Stefano might have solved his problem by now, but just to >> complete this, here is a post by Brian R. Landy from an older thread >> which gives the complete code for -rolling-, including merging the >> results files. >> >> http://www.stata.com/statalist/archive/2009-09/msg01239.html >> >> The thread also points out the speed problems with rolling for panel data. >> >> P.Sarkar * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**References**:**st: how can i make my loop run faster?***From:*Stefano Rossi <sr525@cornell.edu>

**Re: st: how can i make my loop run faster?***From:*Partho Sarkar <partho.ss+lists@gmail.com>

**RE: st: how can i make my loop run faster?***From:*Stefano Rossi <sr525@cornell.edu>

**Re: st: how can i make my loop run faster?***From:*Partho Sarkar <partho.ss+lists@gmail.com>

**Re: st: how can i make my loop run faster?***From:*Partho Sarkar <partho.ss+lists@gmail.com>

**Re: st: how can i make my loop run faster?***From:*Partho Sarkar <partho.ss+lists@gmail.com>

**RE: st: how can i make my loop run faster?***From:*Stefano Rossi <sr525@cornell.edu>

**Re: st: how can i make my loop run faster?***From:*Partho Sarkar <partho.ss+lists@gmail.com>

- Prev by Date:
**RE: st: RE: xtscc and small samples (equal size T and N)** - Next by Date:
**Re: st: Re: use of local in variable name** - Previous by thread:
**Re: st: how can i make my loop run faster?** - Next by thread:
**st: Mersenne Twister method as a pseudorandom number generator** - Index(es):