Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: Programming, rollreg, gaps, and memory


From   Cameron Hooper <chooper@umich.edu>
To   statalist@hsphsun2.harvard.edu
Subject   st: Programming, rollreg, gaps, and memory
Date   Fri, 15 Apr 2005 20:22:06 -0400

Hi

I've been running rolling regressions using C.F. Baum's -rollreg-
routine. The routine is great, but it does require that the data
contains no gaps. Unfortunately my data (grouped by companies) has quite
a few gaps. Annoyingly, I may have many years of data for a company with
only 1 missing observation. However, as far as I can see I have to
delete all the data from this company. e.g.

. use http://www-personal.umich.edu/~chooper/stata/gaps, clear
. list if id == 1

+--------------------------------------------+
| id year y x1 x2 |
|--------------------------------------------|
1. | 1 1980 .1369841 .1180158 .2610746 |
2. | 1 1981 .6432207 .4079702 .1650207 |
3. | 1 1982 .5578017 . .760604 |
4. | 1 1983 .6047949 .871691 .3713805 |
5. | 1 1984 .684176 .4611429 .3795409 |
|--------------------------------------------|
6. | 1 1985 .1086679 .4216726 .9678735 |
7. | 1 1986 .6184582 .8944746 .5823284 |
8. | 1 1987 .0610638 . .7952999 |
9. | 1 1988 .5552388 .6759487 .0907986 |
10. | 1 1989 .8714491 .7152805 .8165695 |
+--------------------------------------------+

. tsset id year
panel variable: id, 1 to 3
time variable: year, 1980 to 1997

. rollreg y x1 x2, move(4) stub(rr)

Number of gaps in sample: 2

Observations with preceding time gaps
----------------------------------
Record | id year
----------+-----------------------
4 | 1 1983
9 | 1 1988
----------------------------------
sample may not contain gaps
r(198);


So to run this file I need to remove id == 1 from the sample. Note that when id == 1 & year == 1986 I actually have enough obs to estimate a regression with 4 years data. In practice a relatively small number of gaps are resulting in a drastic loss of data.

Can anyone suggest how I can overcome (or minimize) this problem and still use -rollreg-?

One solution I tried is to write my own version of -rollreg- that can deal with gaps. Based on help I've received from people on this list I was able to come up with a program (shown below) that worked on a small "testing" dataset. e.g.

. use http://www-personal.umich.edu/~chooper/stata/gaps, clear
. sort id year
. by id: rollreg2 y x1 x2, move(4) stub(rr)
. list id year y x1 x2 rr_r2 rr_N if id == 1

+--------------------------------------------------------------+
| id year y x1 x2 rr_r2 rr_N |
|--------------------------------------------------------------|
1. | 1 1980 .1369841 .1180158 .2610746 . . |
2. | 1 1981 .6432207 .4079702 .1650207 . . |
3. | 1 1982 .5578017 . .760604 . . |
4. | 1 1983 .6047949 .871691 .3713805 . . |
5. | 1 1984 .684176 .4611429 .3795409 . . |
|--------------------------------------------------------------|
6. | 1 1985 .1086679 .4216726 .9678735 . . |
7. | 1 1986 .6184582 .8944746 .5823284 .7084175 4 |
8. | 1 1987 .0610638 . .7952999 . . |
9. | 1 1988 .5552388 .6759487 .0907986 . . |
10. | 1 1998 .8714491 .7152805 .8165695 . . |
+--------------------------------------------------------------+


Seems to work. But when I use -rollreg2- on a larger data set, problems emerge. The program runs for a while producing results and then the internal call to -regress- starts to repeatedly return an error code (_rc = 902). I can't find a list of the meaning of these codes, but experimentation seems to suggest it is a memory problem. But I don't understand why memory should be a problem as prior to running the program, -describe- reveals 95% memory free. I'm at a loss to explain this. Any help would be greatly appreciated.

Here is a copy of -rollreg2-

capture program drop rollreg2
program rollreg2 , byable(onecall)
version 8.2

syntax varlist(min=2), MOVE(integer) STUB(string)

qui generate rc = .

local by "`_byvars'"
tempvar gr
if _by() == 1 {
qui egen `gr' = group(`by')
}
else {
qui gen `gr' = 1
}

qui levels `gr', local(groups)
local k: word count `varlist'
local depvar: word 1 of `varlist'

qui forvalues i = 2/`k' {
local v: word `i' of `varlist'
local vr: word `i' of `varlist'
confirm new variable `stub'_`v'
confirm new variable `stub'_se_`v'
generate `stub'_`v' = .
generate `stub'_se_`v' = .
local reglist "`reglist' `v'"
}

qui {
confirm new variable `stub'_cons
confirm new variable `stub'_se_cons
confirm new variable `stub'_r2
confirm new variable `stub'_RMSE
confirm new variable `stub'_sd_residual
confirm new variable `stub'_N
generate `stub'_cons = .
generate `stub'_se_cons = .
generate `stub'_r2 = .
generate `stub'_RMSE = .
generate `stub'_sd_residual = .
generate `stub'_N = .
}

local max = 0
local min = 1
qui count
local total = r(N)

quietly {

foreach g of local groups {

count if `gr' == `g'
local max = r(N) + `max'

if `=`max' - `move' + 1' < 0 | `=`max' - `min'' < `move' - 1 {
local min = `max' + 1
continue
}

forvalues i = `min'/`=`max' - `move' + 1' {
local j = `i' + `move' - 1

if `j' <= `total' {

capture regress `depvar' `reglist' in `i'/`j'
replace rc = _rc in `j'

if _rc == 0 & e(N) == `move' {

tempvar res
predict `res' if e(sample), res
summarize `res'
replace `stub'_sd_residual = `res' in `j'
replace `stub'_r2 = e(r2_a) in `j'
replace `stub'_RMSE = e(rmse) in `j'
replace `stub'_N = e(N) in `j'
replace `stub'_cons = _b[_cons] in `j'
replace `stub'_se_cons = _se[_cons] in `j'

forvalues l = 2/`k' {

local v: word `l' of `varlist'
replace `stub'_`v' = _b[`v'] in `j'
replace `stub'_se_`v' = _se[`v'] in `j'

}

}
}
}
local min = `max' + 1
}
}

end

Thanks,

Cameron

*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/




© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index