Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: re: st: permutation, strata() for empirical statistic distributions

From (Jeff Pitblado, StataCorp LP)
Subject   Re: re: st: permutation, strata() for empirical statistic distributions
Date   Wed, 05 Nov 2003 14:45:09 -0600

David Airey <> asks about using -permute- with a
nested ANOVA:

> Here's a concrete example of what I want to do using permute. I have a 
> simple data set with 2 groups, 5 subjects per group, and 10 replicate 
> measures per subject. The anova statement for this data is:
> . anova y group / subject|group /
> I'd like to use permute in the same way as the example on page 157 of 
> R[N-R], "permute", to get a distribution of statistics for my data 
> permuted within subject within group. Does "permute" handle nested 
> models? I'm assuming the option strata(subject group) does this?
> How would one write a program equivalent to the one given on page 157 
> for the model above? The program on page 157 for the anova model "anova 
> y treatment subject" is:
> program panova
> 	version 8.0
> 	args response fac_intrst fac_other
> 	anova `response' `fac_intrst' `fac_other'
> 	test `fac_intrst'
> end
> This program is called by the permute statement:
> . permute treatment "panova y treatment subject" treatmentF=r(F) 
> modelF=e(F), reps(1000) strata(subject) saving(permanova)
> My problem is that the examples don't tell me what to do with the / | 
> characters in my anova statement.

For David's particular analysis, the -panova- program can be modified to
accommodate the call to -anova-, but I'm not sure what statistic David had an
interest in (of its permutation distribution).  Assuming David is only
interested in the permutation distribution of the F statistic of -group- on
-subject|group-, then David could use the following:

***** BEGIN: panova_nest.ado
program panova_nest
	version 8.0
	args response group subject
	anova `response' `group' / `subject'|`group' /
	test `group' / `subject'|`group'
***** END: panova_nest.ado

Unfortunately, using -panova_nest- with -permute- and supplying the option
-strata(subject group)- will NOT have the expected result.  The -anova- and
-test- results will be the same for each permutation because the effect of the
following command

	. permute y "panova_nest y group subject" groupF=r(F),
		reps(5) strata(group subject)

is to permute the response values within each subject of each group, thus the
sum of squares from each permutation will all be the same, so the F test will
take on the same value as well.

The example in [R] permute on page 157 works out because of 2 things:

1.  The hypothesis of interest was clearly identified.

	The hypothesis associated with the F statistic for the treatment

2.  There is a way to permute the data that is consistent with the hypothesis
    of interest.

	The null hypothesis for the example on page 157 is that there is no
	treatment effect, thus randomly permuting the treatment identifiers
	among the observations should result in values of the F statistics
	that are similar to the one we observed.  Note that we permute the
	treatment ids within subject, because we do not want to confound our
	result with a possible subject effect (which is not part of the
	hypothesis we are interested in).

If David is interested in the permutation distribution of

	test group / subject|group

then a little more programming is required because -permute- is not able to
permute groups of observations, that is, -permute- does not have a -cluster()-

If David's groups were unbalanced, I would have to end the discussion here
given that a solution is beyond the scope of this little email.

Fortunately, David's since data is balanced, I can keep going.  Since the
groups are balanced (in both the number of subjects but also the number of
replicates per subject), we would do the following:

1.  reshape the data from long to wide, making sure that the subject ids are
    unique between group (it also helps to have a replicate id within subject
    before we reshape)

2.  modify the program that -permute- calls to reshape the data back to long,
    run the -anova-, run the -test-, post the results of -test- to -r()-, and
    reshape back to wide.

Here is a modification of -panova_nest_ from above that accounts for
reshaping the data.

***** BEGIN: panova_nest_reshape.ado
program panova_nest_reshape, rclass
	version 8.0
	args response group subject

	reshape long

	capture noisily {
		anova `response' `group' / `subject'|`group' /
		test `group' / `subject'|`group'
		return add
	local rc = c(rc)

	reshape wide

	exit `rc'
***** END: panova_nest_reshape.ado

Here is what I did to reshape the data when testing this idea:

    	. egen id = group(group subject)
	. bysort id : gen obs = _n
	. reshape wide y, i(id) j(obs)

Now just use -permute- with the wide data and new program:

	. permute group "panova_nest_reshape y group subject"
		groupF=r(F), reps(5)

*   For searches and help try:

© Copyright 1996–2022 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index