RE: st: IRT with GLLAMM

[jverkuilen <jverkuilen@gc.cuny.edu>]

Stas,

Sounds like an empirical unidentification problem to me, just as you say, and my advice is exactly the same as Joseph Coveney's: Run the Rasch model, which can be done simply as a random intercept logistic regression in -xtlogit-, -xtmelogit- or -clogit- (the latter for conditional ML favored by many Rasch-ies). 

If you can figure out how to run the 2PL in -xtmelogit- that would be great I messed around a little on that but don't know how to specify the requisite bilinear form in terms of the random effects covariance matrices. 

For your data, you may need to impose a prior. This can often be done by adding some pseudo-observations to your dataset that fit the model towards which you want to shrink. Having read the original papers of Bock et al on the estimation, the slope parameter has more need for a prior than the location (unsurprisingly). (All this is summarized in F Baker and SH Kim, 2002, Item Response Theory Estimation, CRC Press.)  

JV

-----Original Message-----
From: "Stas Kolenikov" <skolenik@gmail.com>
To: statalist@hsphsun2.harvard.edu; "Multilevel modelling discussion list" <multilevel@jiscmail.ac.uk>
Sent: 3/4/2009 6:46 PM
Subject: st: IRT with GLLAMM

My data set consists of students (CourseID variable), their test
questions (Question) and 0/1 indicator of whether they've answered the
question correctly. The data are in the long form appropriate for
GLLAMM. I am modeling the questions as fixed/parameters of the model,
and students as random factors. Here's what I have:

* generate dummy variables for questions
xi i.Question, noomit
* specify the equation for the random factor: the dummy variables from
the previous command
eq diff : _IQ*
* variance of the random factor identification
constr 55 [Cou1_1]_IQuestion_4 == 1
* call to gllamm: 2-parameter IRT
gllamm Correct _IQ* , fam( bin ) link( logit ) i( CourseID ) eq( diff
) nocons constr(55)

I am specifying -nocons- so that each question has its own intercept
(sensitivity times difficulty, in IRT terms), and the factor loadings
from -eq()- option should give me the sensitivity.

-gllamm-, however, has trouble converging. Does it have to do with
empirical underidentification? Do I need to search for a better
identifying variable? Question 4 above was the first on the list that
had any variability; everybody answered the first three questions. It
is probably not a terrific question to give identification, too: only
a couple people missed it. My sample sizes are not terrific, either: I
have about 40 students and about 30 questions. And there are lots of
easy questions that were missed by one or two or three students only.
If I have only one student who missed a question, then I probably
won't be able to identify two parameters for that question, right?

Finally, since we are talking about random effects logit in Stata, is
there any way to run this with -xtmelogit-? It should be faster, at
least.

-- 
Stas Kolenikov, also found at http://stas.kolenikov.name
Small print: I use this email account for mailing lists only.
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/