[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

From |
Cameron Hooper <chooper@umich.edu> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
st: Programming, rollreg, gaps, and memory |

Date |
Fri, 15 Apr 2005 20:22:06 -0400 |

Hi

I've been running rolling regressions using C.F. Baum's -rollreg-

routine. The routine is great, but it does require that the data

contains no gaps. Unfortunately my data (grouped by companies) has quite

a few gaps. Annoyingly, I may have many years of data for a company with

only 1 missing observation. However, as far as I can see I have to

delete all the data from this company. e.g.

. use http://www-personal.umich.edu/~chooper/stata/gaps, clear

. list if id == 1

+--------------------------------------------+

| id year y x1 x2 |

|--------------------------------------------|

1. | 1 1980 .1369841 .1180158 .2610746 |

2. | 1 1981 .6432207 .4079702 .1650207 |

3. | 1 1982 .5578017 . .760604 |

4. | 1 1983 .6047949 .871691 .3713805 |

5. | 1 1984 .684176 .4611429 .3795409 |

|--------------------------------------------|

6. | 1 1985 .1086679 .4216726 .9678735 |

7. | 1 1986 .6184582 .8944746 .5823284 |

8. | 1 1987 .0610638 . .7952999 |

9. | 1 1988 .5552388 .6759487 .0907986 |

10. | 1 1989 .8714491 .7152805 .8165695 |

+--------------------------------------------+

. tsset id year

panel variable: id, 1 to 3

time variable: year, 1980 to 1997

. rollreg y x1 x2, move(4) stub(rr)

Number of gaps in sample: 2

Observations with preceding time gaps

----------------------------------

Record | id year

----------+-----------------------

4 | 1 1983

9 | 1 1988

----------------------------------

sample may not contain gaps

r(198);

So to run this file I need to remove id == 1 from the sample. Note that when id == 1 & year == 1986 I actually have enough obs to estimate a regression with 4 years data. In practice a relatively small number of gaps are resulting in a drastic loss of data.

Can anyone suggest how I can overcome (or minimize) this problem and still use -rollreg-?

One solution I tried is to write my own version of -rollreg- that can deal with gaps. Based on help I've received from people on this list I was able to come up with a program (shown below) that worked on a small "testing" dataset. e.g.

. use http://www-personal.umich.edu/~chooper/stata/gaps, clear

. sort id year

. by id: rollreg2 y x1 x2, move(4) stub(rr)

. list id year y x1 x2 rr_r2 rr_N if id == 1

+--------------------------------------------------------------+

| id year y x1 x2 rr_r2 rr_N |

|--------------------------------------------------------------|

1. | 1 1980 .1369841 .1180158 .2610746 . . |

2. | 1 1981 .6432207 .4079702 .1650207 . . |

3. | 1 1982 .5578017 . .760604 . . |

4. | 1 1983 .6047949 .871691 .3713805 . . |

5. | 1 1984 .684176 .4611429 .3795409 . . |

|--------------------------------------------------------------|

6. | 1 1985 .1086679 .4216726 .9678735 . . |

7. | 1 1986 .6184582 .8944746 .5823284 .7084175 4 |

8. | 1 1987 .0610638 . .7952999 . . |

9. | 1 1988 .5552388 .6759487 .0907986 . . |

10. | 1 1998 .8714491 .7152805 .8165695 . . |

+--------------------------------------------------------------+

Seems to work. But when I use -rollreg2- on a larger data set, problems emerge. The program runs for a while producing results and then the internal call to -regress- starts to repeatedly return an error code (_rc = 902). I can't find a list of the meaning of these codes, but experimentation seems to suggest it is a memory problem. But I don't understand why memory should be a problem as prior to running the program, -describe- reveals 95% memory free. I'm at a loss to explain this. Any help would be greatly appreciated.

Here is a copy of -rollreg2-

capture program drop rollreg2

program rollreg2 , byable(onecall)

version 8.2

syntax varlist(min=2), MOVE(integer) STUB(string)

qui generate rc = .

local by "`_byvars'"

tempvar gr

if _by() == 1 {

qui egen `gr' = group(`by')

}

else {

qui gen `gr' = 1

}

qui levels `gr', local(groups)

local k: word count `varlist'

local depvar: word 1 of `varlist'

qui forvalues i = 2/`k' {

local v: word `i' of `varlist'

local vr: word `i' of `varlist'

confirm new variable `stub'_`v'

confirm new variable `stub'_se_`v'

generate `stub'_`v' = .

generate `stub'_se_`v' = .

local reglist "`reglist' `v'"

}

qui {

confirm new variable `stub'_cons

confirm new variable `stub'_se_cons

confirm new variable `stub'_r2

confirm new variable `stub'_RMSE

confirm new variable `stub'_sd_residual

confirm new variable `stub'_N

generate `stub'_cons = .

generate `stub'_se_cons = .

generate `stub'_r2 = .

generate `stub'_RMSE = .

generate `stub'_sd_residual = .

generate `stub'_N = .

}

local max = 0

local min = 1

qui count

local total = r(N)

quietly {

foreach g of local groups {

count if `gr' == `g'

local max = r(N) + `max'

if `=`max' - `move' + 1' < 0 | `=`max' - `min'' < `move' - 1 {

local min = `max' + 1

continue

}

forvalues i = `min'/`=`max' - `move' + 1' {

local j = `i' + `move' - 1

if `j' <= `total' {

capture regress `depvar' `reglist' in `i'/`j'

replace rc = _rc in `j'

if _rc == 0 & e(N) == `move' {

tempvar res

predict `res' if e(sample), res

summarize `res'

replace `stub'_sd_residual = `res' in `j'

replace `stub'_r2 = e(r2_a) in `j'

replace `stub'_RMSE = e(rmse) in `j'

replace `stub'_N = e(N) in `j'

replace `stub'_cons = _b[_cons] in `j'

replace `stub'_se_cons = _se[_cons] in `j'

forvalues l = 2/`k' {

local v: word `l' of `varlist'

replace `stub'_`v' = _b[`v'] in `j'

replace `stub'_se_`v' = _se[`v'] in `j'

}

}

}

}

local min = `max' + 1

}

}

end

Thanks,

Cameron

*

* For searches and help try:

* http://www.stata.com/support/faqs/res/findit.html

* http://www.stata.com/support/statalist/faq

* http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**st: Re: Programming, rollreg, gaps, and memory***From:*"Michael Blasnik" <michael.blasnik@verizon.net>

- Prev by Date:
**st: programation of a new function** - Next by Date:
**st: subpop question** - Previous by thread:
**st: programation of a new function** - Next by thread:
**st: Re: Programming, rollreg, gaps, and memory** - Index(es):

© Copyright 1996–2014 StataCorp LP | Terms of use | Privacy | Contact us | What's new | Site index |