st: Stata not recommended for exact 2x2 test

[David Airey <david.airey@Vanderbilt.Edu>]

I was looking at the following citation, and found the author  
explicitly argued against using Stata for a particular 2x2 exact test.  
HEre is the citation:

Int J Epidemiol. 2008 Aug 18. [Epub ahead of print]

Analysis of 2 x 2 tables of frequencies: matching test to experimental  
design.

Ludbrook J.

Department of Surgery, The University of Melbourne, Parkville, Victoria,
Australia.

BACKGROUND: Biomedical investigators often use unsuitable statistical  
techniques
for analysing the 2 x 2 tables that result from their experimental  
observations.
This is because they are confused by the conflicting, and sometimes  
inaccurate,
advice they receive from statistical texts or statistical consultants.  
METHODS:
These consist of a review of published work, and the use of five  
different
statistical procedures to analyse a 2 x 2 table, executed by StatXact  
8.0,
Testimate 6.0, Stata 10.0, SAS 9.1 and SPSS 16.0. Discussion and  
Conclusions It
is essential to classify a 2 x 2 table before embarking on its  
analysis. A useful
classification is into (i) Independence trials (doubly conditioned).  
These almost
never occur in biomedical research because they involve predetermining  
the column
and row totals in a 2 x 2 table. The Fisher exact test is the best  
method for
analysing these trials. (ii) Comparative trials (singly conditioned).  
These
correspond to the usual experimental design in biomedical work, in  
which a sample
of convenience is randomized into two treatment groups, so that the  
group
(column) totals are fixed in advance. The proper tests of significance  
are exact
tests on the odds ratio, on the ratio of proportions (relative risk  
and risk
ratio) or on the difference between proportions. (iii) Double  
dichotomy trials
(unconditional). In these, a genuine random sample is taken from a  
defined
population. Thus, neither column nor row totals are fixed in advance.  
The only
practicable test is Pearson's chi(2)-test. In analysing any of the  
above trials,
exact tests are to be much preferred to asymptotic (approximate)  
tests. The
different commercial software packages use different algorithms for  
exact tests,
and can give different outcomes in terms of P-values and confidence  
intervals.
The most useful are StatXact and Testimate.
and from the discussion:

Which statistics program?

It is safe to use any of the statistics programs listed in
Table 4 for the asymptotic form of Pearson’s chi2-test and
for Fisher’s exact test. But for an exact test on OR
=1,
it is safe only to use the Testimate single-conditioned
option. For the exact test on RR
=1, and for the exact
test on the difference in proportions (p2 – p1
=0), it is
safe to use StatXact and Testimate, but not Stata.

and later in the appendix:


Commercial software packages for
exact tests

Stata 10.0 (Stata Corporation, College Station, TX,
USA) provides ‘exact’ 95% CIs for the test statistics
OR, RR and the difference in proportions. However,
the CI for OR is a good deal wider than that given by
StatXact, which suggests that this is the result of
(inappropriate) double conditioning. The only
P-values that Stata gives are those that result from
Fisher’s exact test (that is, with double conditioning).



--
David C. Airey, Ph.D.
Pharmacology Research Assistant Professor
Center for Human Genetics Research Member
Vanderbilt University School of Medicine


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/