Re: st: xt: unit-specific trends

[László Sándor <sandorl@gmail.com>]

Thanks, Nick,

I left out a crucial part: I need to run it for observations in the
10K magnitude (full sample: 400K, but I also try to sample down).

I just had the 200 / 4 mins as a measure of speed.

I would really love to see this speed up.

So I should make the residual-generation a separate command, and make
it byable (but no egen), then? Any other trick up your sleeve?

Gratefully, as always,

Laszlo

On Wed, Apr 18, 2012 at 7:56 PM, Nick Cox <njcoxstata@gmail.com> wrote:
> If a total task takes 3-4 minutes, dots to show progress are
> pointless, in my view.
>
> -egen- is for convenience. Writing -egen- will not speed up; it will
> just slow things down. Nick
>
> 2012/4/19 László Sándor <sandorl@gmail.com>:
>> Or a quick idea: Shall I write an -egen- extension instead? Or all
>> benefits would come from its byability anyway?
>>
>> 2012/4/18 László Sándor <sandorl@gmail.com>:
>>> Let me get back to this now that I know how fast I am doing using -_dots-.
>>>
>>> Now I know it takes 3-4 minutes to loop through 200 cases while all I
>>> do each time is a trivial regression on 4-7 observations and
>>> predicting the residuals.
>>>
>>> I would greatly welcome suggestions on how to speed this up relative
>>> to the code below. Most likely checking all cases for the -if-
>>> condition when only few would satisfy and they could come in blocks
>>> after a single sort could help things but I am out of ideas how to do
>>> that. Making the code "byable" would at least use some features of MP?
>>>
>>> Thanks!
>>>
>>> Laszlo
>>>
>>> sum nid, d
>>> _dots 0
>>> forval i = 1/`r(max)' {
>>> foreach v of varlist assets liabs netassets koejd {
>>> cap reg `v' year post if nid == `i'
>>> if _rc == 0 {
>>> predict resid, resid
>>> qui replace r`v' = resid + _b[post]*post if e(sample)
>>> drop resid
>>> }
>>> }
>>> _dots `i' 0
>>> }
>>>
>>> 2012/4/13 László Sándor <sandorl@gmail.com>:
>>>> Hi all,
>>>>
>>>> I am trying to demean and detrend my panel data allowing for unit
>>>> specific trends (using Stata 11.0 MP for Windows). I found some
>>>> previous posts about this, but I am not satisfied with the speed of
>>>> the solutions. I would be most happy with a "byable" solution, like
>>>> this pseudocode:
>>>>
>>>> bys id: {
>>>> reg var t
>>>> pred dtrended_var, res
>>>> }
>>>>
>>>> I know this is not possible. However, looping through my ids and if
>>>> conditions is not feasible either (or I collect them into a local with
>>>> -levelsof-?). Actually, with all the if conditions, it is not
>>>> attractive either, let alone feasible. (Or if I sort by id, I can use
>>>> in conditions in the balanced subset, which I presume to be much
>>>> faster?)
>>>>
>>>> Or shall I just loop over a new id that will be consecutive integers
>>>> if I -egen, group- the old id (or do the same with ins)?
>>>>
>>>> I had some hopes about -xtdata- or -areg-, but to no avail. Yet I look
>>>> for some guidance on doing this the right way, if even the simple
>>>> -areg- could have been made faster by "orders of magnitude" from Stata
>>>> 11 to 12…
>>>>
>>>> Thank you for any thoughts,
>>>>
>>>> Laszlo
>>>>
>>>> *
>>>> *   For searches and help try:
>>>> *   http://www.stata.com/help.cgi?search
>>>> *   http://www.stata.com/support/statalist/faq
>>>> *   http://www.ats.ucla.edu/stat/stata/
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/