Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down on April 23, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
Nick Cox <njcoxstata@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: statalist-digest V4 #4807 (st: reliability with -icc- ) - Statistics as APPLIED science |

Date |
Fri, 1 Mar 2013 09:55:14 +0000 |

Thanks for the extra context. We can all agree that what you should be doing and how to do it in Stata [NB] are different questions. We're not examiners or reviewers, but my guess is that the same kind of questions would recur if this were part of a thesis, report or submitted paper. Nick On Fri, Mar 1, 2013 at 2:37 AM, Lenny Lesser <lenny3200@gmail.com> wrote: > Thank you all for your comments. > Surely their is a methods and conceptual issue. > There is also a STATA question, which I was mostly discussing here. > > The problem with our scale is that is wasn't meant for smartphone > apps. We actually developed another shorter scale for this purpose > that we tested along side this. The shorter scale had more agreement > between raters and ranked the apps similarly. The reason the first > scale was so poor is that it asked detailed questions that were easy > to miss in looking at a smartphone app. They were also repetitive and > similar. > > For the STATA question, it appears that we've found the right solution. > On Thu, Feb 28, 2013 at 5:37 AM, JVerkuilen (Gmail) >> On Thu, Feb 28, 2013 at 6:10 AM, Nick Cox <njcoxstata@gmail.com> wrote: >>> I do agree broadly with Allan, whether or not that is surprising. >> >> Me too, actually, though I think both Nick and I were misquoted and >> taken out of context. I know the first thing I did with the raw data >> was qnorm and graph box.... >> >> >>> A wilder idea is that rater 4 who gave no score higher than 3 either >>> never knew or somehow forgot that scores could be up to 100 and just >>> used a 5-point scale. Even if #4 did know that, #4 is so out-of-line >>> that including them remains dubious, although doing computations with >>> and without #4 remains manageable. >>> >>> In any case if the highest score is 18, then something else is going >>> on that needs to be spelled out, if only as context. >> >> 100% agree and that's certainly consistent with a good bit of the >> discussion on that thread, but there were quite a bit of discussions >> not strictly aimed at Lenny's original problem but at the broader >> question of ICC estimation using Stata, which is the nature of this >> listserv. >> >> >> >>> Only the original poster can add more context than we already have. In >>> any field that I know about this dataset would be too small to be >>> publishable, except as a toy dataset to make points about method >>> (which I take it is Jay's motive here). >> >> Yes, that's my exact motive. I'm writing a paper on "small sample" >> problems with estimation of reliability coefficients, which are quite >> common in practice. * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/faqs/resources/statalist-faq/ * http://www.ats.ucla.edu/stat/stata/

- Prev by Date:
**Re: st: Ksmirnov one-sided test interpretation** - Next by Date:
**Re: st: margins after stcox with time-dependent covariate** - Previous by thread:
**st: Ksmirnov one-sided test interpretation** - Next by thread:
**st: network Stata** - Index(es):