Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down on April 23, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
László Sándor <sandorl@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: xt: unit-specific trends |

Date |
Wed, 18 Apr 2012 21:15:20 -0400 |

In case anyone cares, this is what I came up with. (Detrends, demeans, and also allows for a level shift.) And this is faster, as I expected. program define genbump, byable(recall, noheader) version 11 syntax =/exp [if] [in], trend(varname) bump(varname) resid(varname) marksample touse, novarlist tempvar res quietly { _regress `exp' `trend' `bump' if `touse' _predict `res', resid replace `resid' = `res'+_b[`bump']*`bump' if `touse' } end 2012/4/18 László Sándor <sandorl@gmail.com>: > Thanks, Nick, > > I left out a crucial part: I need to run it for observations in the > 10K magnitude (full sample: 400K, but I also try to sample down). > > I just had the 200 / 4 mins as a measure of speed. > > I would really love to see this speed up. > > So I should make the residual-generation a separate command, and make > it byable (but no egen), then? Any other trick up your sleeve? > > Gratefully, as always, > > Laszlo > > On Wed, Apr 18, 2012 at 7:56 PM, Nick Cox <njcoxstata@gmail.com> wrote: >> If a total task takes 3-4 minutes, dots to show progress are >> pointless, in my view. >> >> -egen- is for convenience. Writing -egen- will not speed up; it will >> just slow things down. Nick >> >> 2012/4/19 László Sándor <sandorl@gmail.com>: >>> Or a quick idea: Shall I write an -egen- extension instead? Or all >>> benefits would come from its byability anyway? >>> >>> 2012/4/18 László Sándor <sandorl@gmail.com>: >>>> Let me get back to this now that I know how fast I am doing using -_dots-. >>>> >>>> Now I know it takes 3-4 minutes to loop through 200 cases while all I >>>> do each time is a trivial regression on 4-7 observations and >>>> predicting the residuals. >>>> >>>> I would greatly welcome suggestions on how to speed this up relative >>>> to the code below. Most likely checking all cases for the -if- >>>> condition when only few would satisfy and they could come in blocks >>>> after a single sort could help things but I am out of ideas how to do >>>> that. Making the code "byable" would at least use some features of MP? >>>> >>>> Thanks! >>>> >>>> Laszlo >>>> >>>> sum nid, d >>>> _dots 0 >>>> forval i = 1/`r(max)' { >>>> foreach v of varlist assets liabs netassets koejd { >>>> cap reg `v' year post if nid == `i' >>>> if _rc == 0 { >>>> predict resid, resid >>>> qui replace r`v' = resid + _b[post]*post if e(sample) >>>> drop resid >>>> } >>>> } >>>> _dots `i' 0 >>>> } >>>> >>>> 2012/4/13 László Sándor <sandorl@gmail.com>: >>>>> Hi all, >>>>> >>>>> I am trying to demean and detrend my panel data allowing for unit >>>>> specific trends (using Stata 11.0 MP for Windows). I found some >>>>> previous posts about this, but I am not satisfied with the speed of >>>>> the solutions. I would be most happy with a "byable" solution, like >>>>> this pseudocode: >>>>> >>>>> bys id: { >>>>> reg var t >>>>> pred dtrended_var, res >>>>> } >>>>> >>>>> I know this is not possible. However, looping through my ids and if >>>>> conditions is not feasible either (or I collect them into a local with >>>>> -levelsof-?). Actually, with all the if conditions, it is not >>>>> attractive either, let alone feasible. (Or if I sort by id, I can use >>>>> in conditions in the balanced subset, which I presume to be much >>>>> faster?) >>>>> >>>>> Or shall I just loop over a new id that will be consecutive integers >>>>> if I -egen, group- the old id (or do the same with ins)? >>>>> >>>>> I had some hopes about -xtdata- or -areg-, but to no avail. Yet I look >>>>> for some guidance on doing this the right way, if even the simple >>>>> -areg- could have been made faster by "orders of magnitude" from Stata >>>>> 11 to 12… >>>>> >>>>> Thank you for any thoughts, >>>>> >>>>> Laszlo >>>>> >>>>> * >>>>> * For searches and help try: >>>>> * http://www.stata.com/help.cgi?search >>>>> * http://www.stata.com/support/statalist/faq >>>>> * http://www.ats.ucla.edu/stat/stata/ >>> >>> * >>> * For searches and help try: >>> * http://www.stata.com/help.cgi?search >>> * http://www.stata.com/support/statalist/faq >>> * http://www.ats.ucla.edu/stat/stata/ >> >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/statalist/faq >> * http://www.ats.ucla.edu/stat/stata/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**Re: st: xt: unit-specific trends***From:*László Sándor <sandorl@gmail.com>

**References**:**st: xt: unit-specific trends***From:*László Sándor <sandorl@gmail.com>

**Re: st: xt: unit-specific trends***From:*László Sándor <sandorl@gmail.com>

**Re: st: xt: unit-specific trends***From:*László Sándor <sandorl@gmail.com>

**Re: st: xt: unit-specific trends***From:*Nick Cox <njcoxstata@gmail.com>

**Re: st: xt: unit-specific trends***From:*László Sándor <sandorl@gmail.com>

- Prev by Date:
**Re: st: xt: unit-specific trends** - Next by Date:
**Re: st: Generating new variables with values of difference in 2 variables** - Previous by thread:
**Re: st: xt: unit-specific trends** - Next by thread:
**Re: st: xt: unit-specific trends** - Index(es):