Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: Computing differences in probability of type-one error for different samples


From   George Murray <[email protected]>
To   [email protected]
Subject   st: Computing differences in probability of type-one error for different samples
Date   Thu, 17 May 2012 03:27:29 +1000

Statalisters,

Suppose I run a certain model ~1000 times, but a different sample is
used each type the model is run. The statistical significance of one
of the variables is tested for *all* of the ~1000 models. I am aware
that the (of course, arbitrarily chosen) significance level can be
used to find the probability that a type I error has been made but
obviously, the probability of type-I error will not be uniform across
each of the models where the null is rejected. Is it then possible to
use Stata to compute the probability of that a type I error has been
made (or is the only solution Bayesian techniques?)

Is it blasphemous to use the p-value (such that the differing
probabilities of a type-I error is adjusted for) as an approximation,
given that I have no a priori information? Is anyone aware of any
applied papers which have (rightly or wrongly) used this to
approximate that a type-I error has been made? Since the same model is
used, is it possible that the p-value is inversely related to the
probability of a type-I error. (Apologies if this question sounds
really silly).

Regards,

George.
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index