RE: RE: st: Pearson/Spearman Correlation Matrix

["Abdalla, Ahmed" <ahmed.abdalla@kcl.ac.uk>]

Thanks Red Owl
But even if I use Daniel - corsp- program, I can't get the p values in the same time with the table, and the matrix get mixed when I import it to spreadsheet or word.

I have searched the web and found another program that also does the same:

******
program define corrtbl, rclass
	version 8
	syntax varlist [if] [in], corrvars(varlist)

	local nvar=wordcount("`varlist'")
	local ncvar=wordcount("`corrvars'")

	tempname c0
	local `c0'=1
	tempname w
	local `w'=9
	tempname l0
	local `l0' `""Var.""'
	tokenize `corrvars'

	tempname tmp
	forvalues k=1(1)`ncvar' {
		local `tmp'=`k'-1
		tempname c`k'
		local `c`k''=``c``tmp''''+``w''
		tempname l`k'
		if length("``k''")<=``w'' local `l`k'' `""``k''""'
		else {
			local `l`k''=substr("``k''",1,``w'')
			local `l`k'' `""``l`k'''""'
			}
		}

	tempname tinclist
	local `tinclist' ""
	tokenize `varlist'
	forvalues k=1(1)`nvar' {
		if `k'==1 local `tinclist' "``k''"
		else local `tinclist' "``tinclist'',``k''"
		}

	tokenize `corrvars'
	forvalues k=1(1)`ncvar' {
		local `tinclist' "``tinclist'',``k''"
		}


	if "`if'"=="" local if "if ~missing(``tinclist'')"
	else local if "`if' & ~missing(``tinclist'')"

	quietly count `if'
	noisily di as result r(N) " observations with data for all variables"
	noisily di as result "Spearman above diag/Pearson below"
	noisily di as result ""

	tempname wt
	local `wt'=``w''*(1+`ncvar')

	* Begin table

	tempname dsp
	local `dsp' "display as result"

	``dsp'' _dup(``wt'') "-"
	tempname tmpln
	local `tmpln' "``l0''"
	forvalues k=1(1)`ncvar' {
		local `tmpln' "``tmpln'' _col(``c`k''') ``l`k'''"
		}
	``dsp'' ``tmpln''
	``dsp'' _dup(``wt'') "-"

	tempname tmpR
	tempname tmpS
	tempname tmpln2
	tempname fmt
	local `fmt' "%7.4f"
	forvalues k=1(1)`nvar' {
		tokenize `varlist'
		local `tmp'="``k''"
		if length("``tmp''")>``w'' local `tmpln'=substr("``tmp''",1,``w'')
		else local `tmpln'="``tmp''"
		local `tmpln'=`""``tmpln''""'
		local `tmpln2' ""
		forvalues j=1(1)`ncvar' {
			tokenize `corrvars'
			if "``tmp''"=="``j''" {
				local `tmpR'=1
				local `tmpS'=.
				}
			else if `j'>`k' {
				quietly spearman ``tmp'' ``j'' `if' `in'
				local `tmpR'=r(rho)
				local `tmpS'=r(p)
				}
			else {
				quietly corr ``tmp'' ``j'' `if' `in'
				local `tmpR'=r(rho)
				local `tmpS'=tprob(r(N)-2,r(rho)*sqrt(r(N)-2)/sqrt(1-r(rho)^2))
				}

			local `tmpln' "``tmpln'' _col(``c`j''') ``fmt'' ``tmpR''"
			local `tmpln2' "``tmpln2'' _col(``c`j''') ``fmt'' ``tmpS''"
			}
		``dsp'' ``tmpln''
		``dsp'' ``tmpln2''
		``dsp'' ""
		}

	``dsp'' _dup(``wt'') "-"

end
*****

The results are similar and the table here is easily imported to word and excel. In addition, it seems to report the p values. I am not sure if I understand the code here and that's why I can't determine if what is being reported below each coefficient is the p value ? and if so what is the significance level?

Any suggestions?
 

________________________________________
From: owner-statalist@hsphsun2.harvard.edu <owner-statalist@hsphsun2.harvard.edu> on behalf of Red Owl <rh.redowl@liu.edu>
Sent: 12 January 2014 17:10
To: statalist@hsphsun2.harvard.edu
Subject: RE: RE: st: Pearson/Spearman Correlation Matrix

Ahmed,

You asked two questions, which I'll answer in sequence.

First, you asked about how to modify my suggested code or how to use
Daniel Klein's -corsp- program (which more efficiently and more
generally implements the same strategy that I followed in my code).

I would suggest you just use Daniel Klein's -corsp-, which I copy for
you now from http://www.stata.com/statalist/archive/2014-01/msg00352.html:

**** Begin Daniel Klein's -corsp- Program Code *****

*! version 1.0.0 09oct2013

pr corsp ,rclass
    vers 9.2

    syntax varlist(min = 2 num) [if] [in]

    marksample touse
    qui cou if (`touse')
    if !(r(N)) err 2000

    loc nvars : word count `varlist'

    tempname R rho

    qui cor `varlist'
    mat `R' = r(C)

    qui spearman `varlist'
    mat `rho' = r(Rho)

    forv c = 1/`nvars' {
        forv r = `= `c' + 1'/`nvars' {
            mat `R'[`r', `c'] = `rho'[`r', `c']
        }
    }

    matlist `R'
    ret mat R = `R'
end

**** End Daniel Klein's Program Code *****

To apply Daniel's -corsp- program do the following:
(1) Load your data set and then install Daniel's program by copying his
code.
(2) Then run his program by entering -corsp X1 X2 X3 X4 X5 X6 X7-

Second, you asked about how to report p-values for the correlations and
about differences between Stata's commands -corr- and -pwcorr-.  As you
will read in the help files for those two commands, -corr- applies
listwise (also called casewise) deletion of missing values, whereas
-pwcorr- by default applies pairwise deletion of missing values unless
you add the option -listwise-.  If you have complete data, there will be
no difference between the two sets of results with the defaults for
-corr- and -pwcorr-.

If you wonder why Daniel chose to use -corr- to obtain the Pearson r
results with -corsp-, I believe he did that because -pwcorr- does not
output a complete r matrix in the r class results.

If you want to get the p-values for the Pearson r's reported in Daniel's
-corsp-, you can do the following:
(1) Make sure you are using listwise deletion of missing values and the
same sample in both -pwcorr- and -spearman-.
(2) Run -pwcorr X1 X2 X3 X4 X5 X6 X7, listwise sig- to obtain p-values
for the Pearson r results.
(3) Run -spearman X1 X2 X3 X4 X5 X6 X7, stats(p)- to obtain p-values for
the Spearman rho results.

Hope that helps.

Red Owl
redowl@liu.edu
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/