[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Analysis of pilot drug RCT with small samples and continuous DVs

From   Maarten buis <>
Subject   Re: st: Analysis of pilot drug RCT with small samples and continuous DVs
Date   Mon, 3 Aug 2009 10:37:05 +0000 (GMT)

--- On Mon, 3/8/09, Jason Ferris wrote:
> Stata 10 released Fisher's exact methods for small sample
> studies. These methods are for non-linear (logistic,
> poisson) distributions and apply to RxC based categorical
> models.  I would like to know if there exists a more
> robust way of analysing small sample pilot studies (i.e.,
> 10 people in each treatment group: treatment and placebo)
> where the outcome is continuous? That is, something
> similar to Stata's exact statistics.

Those exact methods exist because everything gets harder 
when your model is non-linear in the parameters. On the 
positive side: everything gets easier when things are 
linear in the parameters, for instance when you do a 
simple t-test. Alternatively, if you are worried about the
normality assumption, you could look at the whole range of 
statistics based on rank statistics, for instance Roger 
Newson's somersd package (Newson 2002, 2006a 2006b). You 
can get the somersd by typing in Stata: 
-ssc install somersd-. If you have such a small sample size 
then your first port of call should probably be to plot the 
individual observations. I like Nick Cox's -stripplot- for
that, to get it you type in Stata -ssc install stripplot-.
The helpfile contains lots of useful examples you can run 
using the auto dataset (the dataset you get when you type
in Stata -sysuse auto-. I would not go for the so called 
robust standard errors you can get in many regression type 
of analyses, as their logic is based on large samples.

As an aside I am not so sure that exact methods are so 
suitable for small samples, as they aren't exact at all but
conservative. Remember that if you have choosen a 
significance level of 5% than that means that you want to 
reject a true null hypothesis in 5% of the samples you draw.
With exact methods you will do so less often. That sounds 
good, but a) it means that exact methods aren't exact at all,
and b) the price you pay is less power for your statistical 
test (less likely to find a significant difference when you 
should) If you have very small datasets then you need to be
as efficient as possible with the little information you do 
have, and not throw some power away because the method has a 
nice catchy name like exact methods.

Hope this helps,

Roger Newson, 2006a, Confidence intervals for rank 
statistics: Somers' D and extensions. The Stata Journal,
6(3): 309-334.

Roger Newson, 2006b, Confidence intervals for rank 
statistics: Percentile slopes, differences, and ratios.
The Stata Journal, 6(4):497-520.

Roger Newson, 2002, Parameters behind "nonparametric" 
statistics: Kendall's tau, Somers' D and median 
differences. The Stata Journal, 2(1): 45-64.
Maarten L. Buis
Institut fuer Soziologie
Universitaet Tuebingen
Wilhelmstrasse 36
72074 Tuebingen


*   For searches and help try:

© Copyright 1996–2017 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index