Re: st: How does the svy prefix work?

[Richard Williams <Richard.A.Williams.5@ND.edu>]

At 07:18 PM 12/27/2005, Richard Williams wrote:
> I'm curious, then, exactly how does svy do its magic?  Does it go 
> line through line of your program, adding an svy: prefix to every 
> command that supports it, e.g. ologit?  Or does it wait until the 
> entire program has run, and do something with the final stored results?
Based on some very helpful messages from Jeff Pitblado and some trial 
and error experimentation on my part, here is my attempt to answer my 
own question.  If I am getting it all wrong, hopefully Jeff or 
someone else can correct me.

HOW SVY WORKS: svy goes through a multi-stage process.  In stage 1, 
it runs your program quietly.  The pweights from your svyset file are 
fed to the program as iweights, and any necessary sample selection is 
also done.  At the end of stage 1, the coefficient estimates are all 
correct, but the standard errors and significance tests are not.  If 
you want to see what things look like after stage 1, use the noisily 
option of svy, e.g.

svy, noisily: myprog y x1 x2 x3

In stage 2, a miracle occurs, and all the standard errors and 
significance tests are now right.  Well, maybe it is not a miracle, 
but in any event adjustments are made for your sampling design, and 
all is well at the end.

In short, svy does not go through your program line by line adding an 
svy: prefix to everything.  Instead, it runs the entire program 
quietly, then adjusts the results to reflect the sampling scheme, and 
then replays the final results.

POTENTIAL PROBLEMS: In general this procedure works extremely 
well.  However, people writing their own programs should realize a 
few things.  Suppose you have written code to display things either 
before or after the model results are printed out.  For example, you 
compute some additional statistics, like AIC or BIC, and print them 
out after the model has run.  Well, that code gets executed, but only 
in stage 1 when the program is being run quietly.  Hence, you don't 
see it when the results get replayed after stage 2.  You'll see them 
if you use the noisily option; but they might be wrong anyway, 
because the adjustments for stratification have not been made 
yet.  (The stats might also be wrong because they are not appropriate 
when svy is used.)

In the case of gologit2, I have at least 3 options that get affected 
by this.  The gamma option presents an alternative parameterization 
of the model, and does not get displayed when svy replays 
results.  The store option saves the model results; but it gets 
executed in Stage 1, before the stratification adjustments have been 
made.  autofit is, in effect, a stepwise model building 
procedure.  The stratification adjustments need to be made after each 
step, but instead they only get made at the end and hence you get the 
wrong final model.  All of these problems result from the fact that I 
am doing quirky and unconventional things that would probably horrify 
the pros at Stata, but then I don't write programs for a living either. :)

SOLUTIONS: If you are calculating your own statistics and you think 
it is vital that they be displayed after using svy, the officially 
approved approach seems to be to use the estats command.  See -help 
estat programming- to add your own -estat- subroutines.  Or, if you 
don't want to do it the officially approved way, you can just write a 
post-estimation command, e.g. I could write a program called golpost 
that was run after gologit2.  For my own part, I am also 
experimenting with a customized alternative to svy that I call 
golsvy.  It traps all my problematic options before svy: gets to them 
and provides the specialized handling they need to come out right at 
the end.  So, instead of svy:gologit2 you would type 
golsvy:gologit2.  It will probably be a little while before I am 
ready to release it though.


-------------------------------------------
Richard Williams, Notre Dame Dept of Sociology
OFFICE: (574)631-6668, (574)631-6463
FAX:    (574)288-4373
HOME:   (574)289-5227
EMAIL:  Richard.A.Williams.5@ND.Edu
WWW (personal):    http://www.nd.edu/~rwilliam
WWW (department):    http://www.nd.edu/~soc 
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/