Dear all,
I have a question about xtprobit. I ran xtprobit (random effects)
for an unbalanced panel-data and found that all independent variable
(sometimes except one variable or two) were significant. I'm not
familiar with such a result and afraid that something might be wrong.
So I am here seeking for your suggestions/advice.
I guess the reason for "everything is significant" could be either:
R1) there were too many failures (dependent variable = 0) than success
(dep = 1); the result is biased toward failures;
R2) the number of sample was huge (if the sample are so huge, the
independent variables that aren't significant tend to be found
"significant." Does it hold for probit, too?);
or
R3) the independent variables are really significant
(i.e. I don't have to worry about the results).
What would you do to examine what causes the "everything is significant"
results (and whether the results are really correct)? More specifically,
my questions could be, for example, when the proportion of
success/failure is unbalanced very much, is xtprobit an appropriate
command? Or are there other commands, more appropriate than xtprobit?
etc.
Also, please tell me, if there are other potential causes/problems
you find that I'm not aware of, please.
The attached in the rest of this posting is an example of the output
from Stata 8 (one independent variable is (eventually) found as
insignificant here):
O1) tabulate and summarize (to see the overall view of data), and
O2) xtprobit, preceded by probit in order to make xtprobit converge,
and followed by quadchk checking the stability of xtprobit's results
together with other commands.
# As to the usage of probit, quadchk, and other commands, the series
# of discussions on xtprobit in Sep. 2002 here is very helpful.
Any help will be appreciated. Thank you very much.
Regards,
------- Stata 8 output from here to the end of this posting ------
. tabulate dep year
| year
dep | 90 91 92 93 | Total
-----------+--------------------------------------------+----------
0 | 34,480 41,974 49,580 45,682 | 171,716
1 | 236 804 506 2,834 | 4,380
-----------+--------------------------------------------+----------
Total | 34,716 42,778 50,086 48,516 | 176,096
. summarize
Variable | Obs Mean Std. Dev. Min Max
-----------+--------------------------------------------------------
year | 176096 91.6383 1.084519 90 93
id | 442352 111200.5 66353.34 1 239086
dep | 176096 .0248728 .1557379 0 1
x1 | 442352 .1626749 .2227833 0 1
x2 | 176096 .0290038 .1161555 0 1
-----------+--------------------------------------------------------
x3 | 176096 .2804436 1.420482 0 68
x4 | 176096 .0020983 .00406 0 .05
x5 | 176096 27.28332 71.9288 0 868
x6 | 176096 5.703616 8.440668 0 121
x7i | 176096 6054.225 15543.99 .33 135696.8
-----------+--------------------------------------------------------
x7j | 176096 5071.261 13000.79 .33 135696.8
x7ij | 176096 3.03e+07 2.21e+08 .297 1.47e+10
. probit dep x1 x2 x3 x4 x5 x6 x7i x7j x7ij
Iteration 0: log likelihood = -20504.706
Iteration 1: log likelihood = -17467.585
Iteration 2: log likelihood = -17232.621
Iteration 3: log likelihood = -17231.582
Iteration 4: log likelihood = -17231.582
Probit estimates Number of obs = 176096
LR chi2(9) = 6546.25
Prob > chi2 = 0.0000
Log likelihood = -17231.582 Pseudo R2 = 0.1596
----------------------------------------------------------------------
dep | Coef. Std. Err. z P>|z| [95% Conf. Interval]
---------+------------------------------------------------------------
x1 | .9626477 .0273679 35.17 0.000 .9090076 1.016288
x2 | .8126779 .0407537 19.94 0.000 .7328022 .8925536
x3 | .0460465 .0042701 10.78 0.000 .0376773 .0544157
x4 | 3.965495 1.625142 2.44 0.015 .7802758 7.150715
x5 | .0016754 .0000945 17.73 0.000 .0014902 .0018606
x6 | .0122497 .0010104 12.12 0.000 .0102695 .01423
x7i | -1.65e-06 5.22e-07 -3.16 0.002 -2.67e-06 -6.27e-07
x7j | -4.40e-07 5.58e-07 -0.79 0.430 -1.53e-06 6.54e-07
x7ij | -8.84e-11 2.23e-11 -3.96 0.000 -1.32e-10 -4.46e-11
_cons | -2.467827 .0119242 -206.96 0.000 -2.491198 -2.444456
----------------------------------------------------------------------
. scalar ll0 = e(ll)
. mat b = e(b)
. mat b0 = b, (ln(.1/(1-.1)))
. xtprobit dep x1 x2 x3 x4 x5 x6 x7i x7j x7ij, i(id) from(b0, copy) re
Iteration 0: log likelihood = -17297.277
Iteration 1: log likelihood = -17197.392
Iteration 2: log likelihood = -17195.744
Iteration 3: log likelihood = -17195.688
Iteration 4: log likelihood = -17195.688
Random-effects probit regression Number of obs = 176096
Group variable (i): id Number of groups = 61999
Random effects u_i ~ Gaussian Obs per group: min = 1
avg = 2.8
max = 4
Wald chi2(8) = .
Log likelihood = -17195.688 Prob > chi2 = .
----------------------------------------------------------------------
dep | Coef. Std. Err. z P>|z| [95% Conf. Interval]
---------+------------------------------------------------------------
x1 | 1.030274 .0315317 32.67 0.000 .9684727 1.092075
x2 | .8954573 .0467279 19.16 0.000 .8038723 .9870423
x3 | .0452287 .0048709 9.29 0.000 .0356821 .0547754
x4 | 5.102698 1.811342 2.82 0.005 1.552533 8.652863
x5 | .0018586 .0001094 16.99 0.000 .0016443 .002073
x6 | .01222 .0011133 10.98 0.000 .0100379 .014402
x7i | -1.46e-06 5.77e-07 -2.53 0.012 -2.59e-06 -3.26e-07
x7j | -1.41e-07 6.20e-07 -0.23 0.820 -1.36e-06 1.07e-06
x7ij | -9.24e-11 2.55e-11 -3.63 0.000 -1.42e-10 -4.24e-11
_cons | -2.634148 .0256348 -102.76 0.000 -2.684391 -2.583905
---------+------------------------------------------------------------
/lnsig2u | -1.992943 .1386826 -2.264756 -1.72113
---------+------------------------------------------------------------
sigma_u | .3691798 .0255994 .322266 .422923
rho | .1199458 .0146392 .0940842 .1517256
----------------------------------------------------------------------
. scalar ll1 = e(ll)
. quadchk, noout
Refitting model quad() = 8
Refitting model quad() = 16
Quadrature check
Fitted Comparison Comparison
quadrature quadrature quadrature
12 points 8 points 16 points
------------------------------------------------
Log -17195.688 -17195.688 -17195.688
likelihood -7.470e-06 -7.112e-09 Difference
4.344e-10 4.136e-13 Relative difference
------------------------------------------------
dep: 1.0302736 1.0302736 1.0302736
x1 -5.028e-08 -8.018e-11 Difference
-4.880e-08 -7.782e-11 Relative difference
------------------------------------------------
dep: .89545732 .89545731 .89545732
x2 -9.534e-09 -4.996e-11 Difference
-1.065e-08 -5.579e-11 Relative difference
------------------------------------------------
dep: .04522874 .04522875 .04522874
x3 4.139e-09 7.031e-12 Difference
9.151e-08 1.554e-10 Relative difference
------------------------------------------------
dep: 5.1026978 5.1026978 5.1026978
x4 2.846e-08 4.266e-10 Difference
5.577e-09 8.359e-11 Relative difference
------------------------------------------------
dep: .00185862 .00185862 .00185862
x5 -3.218e-10 -4.060e-13 Difference
-1.732e-07 -2.185e-10 Relative difference
------------------------------------------------
dep: .01221998 .01221998 .01221998
x6 1.365e-09 5.915e-13 Difference
1.117e-07 4.840e-11 Relative difference
------------------------------------------------
dep: -1.456e-06 -1.456e-06 -1.456e-06
x7i -7.990e-13 -4.308e-16 Difference
5.487e-07 2.959e-10 Relative difference
------------------------------------------------
dep: -1.414e-07 -1.414e-07 -1.414e-07
x7j -3.279e-13 -3.379e-16 Difference
2.320e-06 2.390e-09 Relative difference
------------------------------------------------
dep: -9.237e-11 -9.237e-11 -9.237e-11
x7ij 6.284e-17 -3.258e-20 Difference
-6.804e-07 3.528e-10 Relative difference
------------------------------------------------
dep: -2.6341479 -2.6341478 -2.6341479
_cons 1.010e-07 2.045e-10 Difference
-3.834e-08 -7.763e-11 Relative difference
------------------------------------------------
lnsig2u: -1.9929432 -1.9929438 -1.9929432
_cons -6.447e-07 -1.340e-09 Difference
3.235e-07 6.723e-10 Relative difference
------------------------------------------------
. scalar lr = 2 * (ll1-ll0)
. scalar p = 0.5 *chi2tail(1,lr)
. di "hand lr is " lr " with p-value " p
hand lr is 71.787482 with p-value 1.198e-17
------- Stata 8 output to here ------
--
Kou(Koichiro Okamura: [email protected])
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
