/fkkTg'QKn!s8Cj/x\!=])2Nicn*^^znN9 ?-ԒSCNP,?\i̇\~$;g7UvFE;eYpV`tC;U_Iap sskЃz7)٭phW;ݛ^dѕNPWAQg#gBMPdY4 #\UҰ 2Fh/H$*`"K{A]5)#~oߑƜ2W6e;f/Q!?}jkHzOh;kKw0&"bY<J!Y &RП$I2H!H21lKD(C@0C"$xoT.YN9 $JtU?A>=@v&vi2uvC2@F6Fdns7 ?kE@r|_z::e,'?ҥt xNbMLh#F\֙kjLW&r5("h7[]aEɟ&q4Z`w{*H>V_,//όNqEtd=藃0NԮL2䓮7@Pѭ*fo3ژ榼 yPq*LQPA9 0R;@ >ȉ!p%.lx* r?a eGy& 'r'J@gƼyuV+2wMIG~!nZG^ͨ}cTK鴋[5Hc)]s,)! ӯ^O/|Cƒ,'Ɍy=m@$36;t!$644PZZe0h~(p X āΉO|Ov?o}1gɌɰk~"vHZǫ,3M:튯-# /,ǌVR쩌7,韙2iiiPN$>h2f3VKC(Ϧ BIda D /NB}rN9 $JW\V|}mu@M_#I߯@d~LZCvYW<~Z]/Xo^D(kn!6}x>raߓ.cb}-H( (1kAmp{wlL[(qWB;g`aMu7 +٣7](!Xz_(T_/H,۫7\$db'F>VSIY?HÁDj8 ҘrToCwq!9l%TFaņ'X^ܭrIUIݯibd?Oq`SVmӻI鍽P}vݟةXK#L1mq]9iM0ݜ/?nG8Ih.ȨQkQ.? ~Đ~j,o&Ist7I2@Y9a%cqqks'Ęu@Z9=9RZL Pܺ`L2͚fYٹз5 Y.}Ħ?( mj @wW|'#K#1#VPP` j+-'@X <j'@<,xwߩ"ׅݒ!DD x厜rI_-R^UN9p`ԕ֠^ SN``Ư2 JpjϚ 5\v~B`XP.rq Xi&Ku661g9ascCqwOizU]"%p(bzLCM*9d`\&XH⿅.))rS]tz,.eepvNF87^duu7ׂtCךF_ sd ^@' k`d;#aWNDOM8#d0PRx7,T3iŁt':Sa8A{$@N.pN18;ldLǸ8vxS*ӾEqNV g8τ%sE[N,PSzgQȭb+53W;&tthIm>E'#Xu:x&YY͉ax8^VVQ J_7' w]qMm-!'oF[N&+W@c&CNkR-hXOzRL T[Nn+YY?̑W`bvtje\wvdD3^=^@L BPT$G*H5Oad-uۂ9KN6j)#qDC?w$@NY)F`K`v09azԹR]d+dh#Dd\)N됕2 emc!' ض#s2;-nBM%fpTr;S~3x]cbdH V% z~@P~ dL7 EG[9^H`;C[;$9455 9 b:8O;;Q8+XT75Ȑd~uGưqYK,%xǓ5c 5V@O~`/ddwj*Ws1ydAt8/̋(q+ʥI 'oHHl"v_Lnw4LNbszi1]kJ7#7Wnҝ;k02 Q#2/XVgp0[鎖s6S~qd(W'sn"kPkN£ L{*yQࠫ+Ѳ57l@\]]&mqUV_\>94im9e yw|IE7Hoc+v-<|B0G KoraHDU@.z[j-@@P w W2"|0JXn4-h<.(܂Q JURGE7r1W(밅0`AJ7pnbg@#)8J4YZ>sWy. t#@[H$+1a|N~ j,>_N,;kiHK"X; W$bTx dPdBe 71c<ʇoJ ?C 4Bç@OP>sJiW_c RJ CHFcg< vp 744xA]]U\̧dͽ' ]C` 0hV?0Z Ǭ} Bua䧫*禤+[t±W_&v=bszggwELj]Y9G{d噾Djf>3ʲG_17+++]7AdbԌƞJiKZ3N[*VXw5qv/4Y[n}Nf0*blNuܛ!V%/$a*39\w%$ZpJD *KYfMPP!pE4VF:5{>}bN.)un`7oy 06xkn¶/c b×~m,}w5vvv߽tˏo]~QSZ(w~*=ջ>d{MwjlXWt/m+~sk|˞_߽wwkh}j[/O/įk 7+XՔɹOPU|CA^`8WUS:usNY8gi,>n &GϭDO9kr =o=;#TC[0/~;ϟwիN/ΘRyKN==/O90x:v^s)_HK$_ ֦|SNs|g}SIeo 2_qKuќ_|yvs>}.^wSg7wr酧̂&<娩ϛ|Ήw7~ocGrԏ9o^z۷XV5A [fZELug0co[~ctIO<~*_c@ܢ&$ߍ 06mdu_] Sn3XC<'n44fԗ!9e0h=pͤ2PJ+kYϐAeGs]tT㓬zHoD.:rڳ?pգV]tnh&)eFx97kʨk_N~k \)&[W.שmyKژ~a~sojm='zW7K`:4CΞs6^hIr)XEaPWÒ)=E6M(.\$R?PUsh` W퓠ab<к_t+|Ϫv2y?h"&^䁷?oxkP+PS6ovxy(JAʲlkѼHPm_W{Wrԑ־kMXYW]0~O?EBFx B|5?7 <|䊣!W"V̞Vh :#?Σn{{:KH"Y]6Ħqpqa.\n:(F}/reGa'9᪪x(Ƿ/:3>Xy"WN*)ȟXSʋD(J ¬`)wZ9xe~9 Xz*SJEnAFGhL'}fap]xU+PƄts[f5LLt@tG`5܁Qks~i7t?qeӚآOqB}̯je:e唱妇GMڝ+{AIFuJެn6Z[[qy_jW%f?srv.O~![։th=k0vqO*ưZn]x9$uekZmUR(U2;I|#Oj跂^(9#ZBB&=u^Oƽ7Tqq Tr'╍!Ҡ 9G.;oK^^' +.Ҵi=~o6V*[?qfD\/y߶}G[wC ?[ݻt۳kE IFBE* ?R_w?-WW^p!fsO/vr3vGgզgVaS#;)* P/~ܼYҭ_W[Vxl,oшyZN {bγOAl'Cd'Onhӟpk%lO%b3C e;E]`M'ϚXcwvn֯'OP8EN &S2M rs7@yEi&0gv`)cg1]1g3Naъ0&x2dfg$q]5b[YkϳV ;ssNHSt'i2D}a6\@$u3!Ȣ2-zih2WF[_rlͯu1I'. 7VF}ypVX;uG!vB-;bFR# D{ H4xO׆ɟdʲW :k/8@\ϽuYnq % f ܄ q<L EXAo#$ Խ'4BP-B&*\d,QM2~ǅ/>кm-Ex1ӿa}qk---n&u '_cz vzcڒyqח?#7{'nCyMRPOT$!M]o:l=-K !KpPjxNE ab3as>veխ:Ve w9W3n{jǻ/;R1XL"$$)eXcx%`$pCk}vrEwpLI]qQ߅l#,ДO+n{` GL(YOܻso3``Q4˯=_IԂmUX]zN8N:]e,;oLs,vbOTd$Xipʲ%Ehw'?t&yqsaT9t MYnA\[G[Q+[<-+V+ek.B,̝@lQ~6K[]b ωΌ@E"]vo cL1[LZj,)I% F 2XSzi)9⽁BW,Kc%C40P4;@|=CKZF^ k3gM+EfWxv7G-͙RyX(M%ao=S:3A1vFc*㪓[چ:H;/.W}ç>՛ dI ayYRB*W3^7}Ӫe偬G7懩5%v40FӖw =#:\ y9|m_S*-|*I$)!yxynco]>:f^X ]Kdqƣ(~'vG~'wa ǯв],klus@쯏mQX5qMμU~h}A>E0#?x[9 $H?6°Q\cLu_nwe #, =̑xHC]Icl:j,Mǀ)Edk::ᬙeJma%L59g8\2ob-9Y3 +ˊ3J@^Oňj@KJJFdWCv&|8q5"٨ŮUšq Ka3$иl hP;J\ F$UOBU KqcWiNξjxBG=5ueKl9]$5ܝ? mНK쬘x%ɚ)D}h`yi)T@Xh$ R# ͝@,㴜;?4̞0f̘01g:P0s(} d4_rS5Ӹʣp2. vg{glos?4-M} =1<_xw]{{ۻZЌ@UK7}ׇF}o9x`j8(, " ƭ:w4`yuEfj(ܷb]{zz{4uq免{SW^|go<_<ͳ{{WnjbցdcSBݱM݃:m߾NW/njaֶŮߺCju. 'SllͽkZۆZ{:_-p݇O;inąL316%n֘1_~$/ï87 6ZOG;wtf@"5HHQL:@w-.Kbth_T3Ty|J=lM R@ԎKJŚ44ih{b[Z9%\ym}qX8$ϬIU'}֬o>/6,XYZgϜTFp [^2mbM]o|GWo(+xc܍\jԶ;wf{K[ZY0}RiKO"u-}J{&Q/۵sġ+"QbB%,O7=KXߴtW?|JMU]m>mXEytj])ƥ h/{]Ew`Ba~4z3;{wS*z "5Ս+J̜\1g-߰G7vr8I,Yn eÇͪ-潴efh1Ɖ*zWϟ]KH =-ݼjS3vmmx^3[fcFKcvViеvbیsV<~}M͞RoQ@NT 8=g&ިS-C7+AF0KuOY2s7˒tuaЄAڹb؎:;9 4nYhC?p iq?.:q1%Od˅t%$(nnn槹cdi&UIƶN[J0P]]m%sHH#٦d\P DuSZ:kzTe9BxN_Ol:hOZHKj5m{lYó_\֨oٿ£cT묌ǂalXOIѩy !ȑih.q@f|+Kn25!Pp;@_Pb@wIo])hXDޞӿB!+ٙmoYx˞btF" \TG2+]v&=da o7=$V4JNbץ4Lq,d\V'5 &;?IZ-*-@jjPZ1W"CaT7,S!1قhl XOtn^6bL%7, Z/tI|>@|$Ѽ|*̒F>jP E[kP@3sOW%Uu+M 䎜rJL39(H@2]B SA8٫?-*_tъD,w.j[c{ y Gk@c" mrI '?681"4y,2w.P14vbK/q[6(s>-U1"+9 NZJmfŁODVH] www~D&`E@C.s"Hg*Ye6@\,نWw\!4XceM ܪ^m7/mmI^h0wlY}2PςhIFC]eG ;9ܻ;W={ޝk KJ˫ugo5$DfKW`&ݭ- KذxQ\TEKrY+*V,jO*=&oQ+@)U݊fogG0CF_]$Fɸf PUق%<0.0;$jq t#e@&A9ጥ~ZX}S:/LAPs *U(L]O5j#"4Lq(9`vuxD|åXy30#yդ8v@;)J\IpRLcX͟OF)"KurKTӚ %%n iIpr+PNSĩ>B)BJc= i/9SADs>^=FC5)SHX"yPf3fܕ9see0ӛYU1/1N,[eVaCT Ys,0Ș*Q "]d.Ydf`͡ hqB\74 ʊ!n/ZM"G)hLNfM2&U.rf qkٰ֖-k_g HObY;C ؉ZO?akya 7ɭxbeú?<|/IZmXjKiknY'qYS{$b#WzIyQ;"$`IeQazY` * 0 ջ J 0((٠)@ '@TAmK CUZC @vI~Ddc"MO0%hA|JYPdҨ42,Q-Egr=7AڊUVT[wN*h 0%j' &3 ;#)i\ZL[*ks$rZȈ"%QI-AQ+ `RHBhDc#[)RORo D@)\5Qu6ȳ5U)P| Յ'6&AҹUϥI_.4Xi_#1 5G~7c/~FØ\7ʜ699)u`vD*Sl$H,34Cf<+Ȅ@*@(VUUyYu@}$ !8![RX-nK8;ah̀k"#ʂ!2%cj.,*-.Rmi;{1~pic#;[%ñDoֵ4?PUՏ\;{J_OwQ3KƏO4(kz tdy&)zf$U2H?*W$&g-}d :J-h .?:OxDрBOS&QUJ(2ϓ[X.:,7-[b%4CyD/jt2C%de"Ԕ6G5* 2ԪӴ,D$bS1mց%hZ$E"s'Tңܬo%ӡ/SkjCՀmzXUTUM땥.bV7j^婉WHStAV\I 'LE^]bA%~2N'E0.%-14$f@a {P8P0(D^I:G!>'r$0s^ה,W ̟!mI.1hj_0yEEWL~kT{,s`^^OQvHQQIggb FNS@mf1nX(TeṔ4J,r&Xlr0 r47V,=yR@_oOSmׅTeyCC]]5ڊK$,9pP}$oa1Rmxpiuutvwvb6C* *Ug+ȇf0ESfh eGD,JEͯiزMJRwnixȐF@D,I١zwlz b2m)S֦]V?o}WoyɁ.vR,Qg;c:,F9)"H*>>*OG9%0)BK`)_$KձvV|$@.Yb2%I 'rH%Z`2y^!_26]՜+)ŴjO9=MVoK&QX1x#-<#.J(?х` (Llh8W+%Y'.(F!M9aq2\Iƿ0JP PyzAQ1*eAm+eK_B̀]Eee+4phx~D{{Jt}ZH/[qLzuZ}CƆ--Sx,.k# j Weikʕn+K@N9 $2&~gFd,ŴOSZ0HӋqF٠*Fܰ5قJ°hLYp _].9H\eoӵ ^~rl6eeW_$#?+@h ~A_<`@W5~c1j2wHev;idSKPV1TfΝ5.@n0a`$xJI6nY[!@nfM6ޱiR$BB ʔuF"K)Z5۸efǦg`簬KJ&^\Zæ/yb;G9ht~Bƪ!9[4tTb#sxҙsO4C*s1^Ob&"GATchuD]gLH.͒~b "g9ԕћ9 $Os<绺A&-fv<]e©6\#er289/Aѵ$@E}=(}+_RRqA%gpEITtReJbv+wDjdoF쯬B..f6U`qrKtwK|} DLqb^/P`L8l]@l> 3-ġ'cTHglB4rETn(oiܶaՃu϶٪Bz;[^nܻ IUEzjd8!^Q>_zt ZES娈`Kuu{lxk5imxϸDG]t⼅Θw=yCFpz/u1E3;SJD^_:dI~CnԬ1cOJ} )u|rE$@N0`9?my5J*?MUImpm0ߪ16]eb(:44mvF3kH2$ۢ"XLO_ȋ!$"ʬfb%'L:M}0PT7F"HuvtTWU>ʜƠAn:ٻ` Mʉ@Ct]رH =FJppQ6:YHIB1,Mήћwnz;#kFudo+ƒΎ UuSX=}ą,-+$bA=% Ibxɢx[Ӹ{ ACdo%JK.:ٲsgut`KiƆC̟ q~#3g~Jz9 UEP-AcW`wœg:w{$'OK Lv`Q-5:q*#W~ۍS{hM#dHu8xְb!pQuUL:4@8ɳM=#`mSEANHip]pgp[SL/"C1Lc[<=գ\w7ˢjup|o^on[7"%Om)ba?鋢E@ߢb%DÞMvO_ExuS"n #D G&4~xl}?~.}5iҼDm60SQ1#XT>zrîo2$64HX :@DO!T-xlzyl7I=,Fry0nˍV8/.>ŖF(PVԕӜ_p6.c(qKRX*z(403&B;0ClR.3}QA`Ě%UdNf84>ҏp:$8)⟂XiTds0t|+&QO@e5gC:'+ӻ>g#CcM{ |c߁Dte 0wU^.kCak=3qGsK\@Wdpl:h3f̠2Y@WlyN']cj8+olx(5ڸq̓}-KJCxtг*mCLUj"}UtjڸG?lXPOgS̈́-U҉9Zۑ.DUX*4 2[vg_o'5'aEPȠR/uٱbzZmiiִ}˪xB'9 +'݄KQ&&:[qv`z"#f.z.`K2ޘ+c\c`MSR~yP"{<*4O v/RZZ^FXd-+4C{VuW=ҜK+# ';OD6WQ\Ҝr.73͜&+`P7Kgeizc]v̚=Ő6f~!3j բN Ã _!R8ICTuXv}=}CfHBYQX)/-&8ʜ3?:gfmđK^S5Cw=^`dWs2UUPyc$IzQ7@!WS\ڜrX^~y.=ȕeJ#[SpZz*;SQZqy 4̈tX Yp|XlKJyF0U<2li$'Yo['+sG&Za`[Μ9Ӷ42A80P@M_0CmYw`xbI\NXO0/2mm;;#S 0<_Wut"65D8O7N&R%e$Cm/>#?okRT\^U77}zԹ MmS&M%= seM9O, D@m۶ql]xYGVݵm#A٫ٲ}\_)^ذK8mW~k'!h~AmRܨ"=f`RkL]o# $7~>y~/ a@j'?reߓ}2Nၰd7_yo2\ӵr7*bN<eyX0O`9 Õŧ|.䬳(*`Oψ~aru -G_%Bh2JPAT)?7wO|ݛ.xk[zmS.AN9 $0StXjP`] ~~}o3fq Q/c%niVHՖ-/ɶT/af-Z6<&;f~~죆yXCX! gI/ur_ :04)a4.[5ޡWt\ZyI5-a8g$'WX2C48$}7;lɥq B2X2i*O5ߝu=Z-H-u!sŐgJW遳t(x==$"BXdmd1+\?s;innawpP^=jZ*F'M [!PXjވ' u?uw 78VA:ݝM/7P5a~yE Rٹ7Zwwܰm}g9(- [u?~jWW`oK"VF(&:%XMvBa%CpSULZT;D8KgW8[&1\p'F.pE(Btw&c ,D[hWI90Æ. V52^9b{c˧] Ŗ}@~]NK˕^cvԁHB$CaQdͮ ! VX\*MVj0O|"mALӆGBJvF% Kd1G=L ]T/\T((OL ֥i2 |'z%xbqN`v;rI 'DN N뜇]qzN&1̑@]8wmvΣuW{Ј.,pXcP>D&.;L[{9GoQm16A,[AQH_J`X'sYlH];k{f8\ZBHȾ=T*TWTGfa<`8x4[ZvK`js۹|_zq ۆl% ]]UQ\RKK{- B!=p>HQ>,J, P){WVAwI*ds$;Tz'͠ȄuAkƃf%}`1bܲ"3 ~-<&/mWniJ%ߊ%.@+ n_{y1&,d~$^l bd((_?_Xmh2 LaR*Ppnj`Oj0 P!c@@)lx:xM%3Os$@N-lpmr]z+s,@k*RclvKr8M4cVL&eh w?3试ebMһ*ErA7w\/XK@VU]q٬d!!G ӏnoSU1kSؖ'm!Z[4 a@Xd`bhmU5==]*P;0qJxHgW{EYe(imig#K*@H @EQy ɉrEf:iUY WT$6Ķ6! x-2TdE3Pn:KK*d \[ވ̹-S5.9r;:z߶1 `S$_b%ZאϨ9o\yww?k;G`Ɣ$%䀑=堆"ߕ]G*Ș 2 39PV%bBkίz]KS5ƅ~!IK(Dj*<ǟC]fJtzK| 7qk:S8wizղ;!uOA*;CKniLM0:.Jc=%E57V-,`#b TUc31DdD=0&(IbAW&=q։f㫭?섳>P\99??M]d{z@`E Ge*j,Ve`qc儞"6E c|h2OlIT+?zᢥ?ί#X @5ॴecoA±ZR==Mgu K5`M#܊¢N5=p'1UJbwd05bpK"?x3?ȪՄrI%`8d zBp.6NV,g35fxsȃ@yNyfy_xܦh"$rLZ}}ѵُ,xN+綑(MC"1 IҼ8Zsεk=Mj%UFqF#p+K.V"4) chH6xQo0/k߸'KC?~ 5tc뚻[>?a"?47Վ44tt̙":j.BaM4M?xW[xl= VlQϖf;/YBɊN6<fQǸE6#R*c1WQ )_xwݵlrE!M wzw_rFi9*e'8L{!鄹~}gO$dO$!a}^ݺ"}g%F%'Λr߷yshBeuLAh h{6nưŜaWc5~g/t1KbBZ=Q^>|:kŞS=\n^F2"ebVZVkK>Kj!ԏ&f/:լq=q̑%uىAզ9qh`TG({ʽkܕA{5)^9O^X@ \JhuȘP,}˕63Ec'Q1=XoH*So=.TjHȲIr?g'.'[CG*Ҝ4u bQ,,+`` I/`p {Oٽ#>%ي2tIXϾ ljnǹ!>EE84q>lng@/Syb4A_qKxx~36c"[=],̋aXdZŚqwH Cl8 Tj86֓4LK$مh.//ř"O _ vKC̟2TI"=pS&M6նO)&;kjdN G*OJ/ĮG6m۹K}o>L8=)^?3cvOCcU)D#צ4.џMٓ&e_EJ,D BAIF1VzY7Atx"b*ӹY{J{i[ R\>ad?lWa+W@aKcbpO6C0~\-Ԋo&Jd>hlKP41YFSURZ8W\R@_X 7smE }0+$<ʈ.a#zKp1/q:)34{,&`Z>:J:G'_HW2[!?gLX?*`4.}C:{aqA}M<1X|Ul~ʸboI?Gdزw5vn.1)8jGX-u13'Q>m%/2= G1:LXLcp/fiJh>\pOy{)%+}<{2Q۟M2\ƃJsuq̄Zү3o( $XYf8CJ6\JBТTjF cѢZbC1VA%ad%%pUs$gAvT**.OVknj0 m"RSS#(1@F)~`bUT:4ozŋދSi$naMR+"L*+L足ˋJLϫ:ŏOiL^}r^HfN1ZoA*fXcP₈M sk`:j>~һ~?LdnAv4#1Z>+R5eK_G1@jڂHW``.qRE[yd<:@R5X%f( MhFd>, [^!pEto{*Egص* DFR$6/uu$R͍XZzL3\ldψ4ǤĞb)[àMq4"hyHXu $ɣkȈ|sgK#6۟a)37_qLe ì~y-+AEt<21$aqN^8EɀOu;n'iַ}S*Q1+bdܵf&~]`HA9- OA` ZjxVD*nSgoUה5@4H,&ҮnU`q#csƢb2L=_=x^E"Ɗ쬓aHĆ ~z2uٌ̒/mpoiF)f,DvA6M@[mVVtg4ӵ.[V.Rfxﲟm}7ǅRb^|_vv)&-u J5;ײݻwi~t5;]jJXu;^٪^*c_vj8sH6ؒ_4DyxG^غ]OKn݂̅LǪM?{O[uڳ[ްwPԼΦn{b6~(5;Xw>qfaЍ|(^]=Ï{O-ݡkgckwI93i;w),{dA̦Nl@\1`\yc6:N3'A**=躬F}L#+N3%:HTorUGy#k xU:)S`XKFdiz?&7jBv˨ >L@@^2}'*-)MI3]Vj=2 5r8z.z_;@qx&e!?xX|⽢A"V(n_O|@LUc^8%KZs]yGYsgqp뗿%/n_\!'> ʶm?#i ~Q3n %ebx~¼_fD+3^ (-#8,k}9 Mfs(ix$tsNkS%'b&Kgk_̓$08pU%.A><(/_[ Glv/.㞭++*;qu<ϚEded1>rN<3VѺ%Fǜ#&0֊DSsOaצ'eÏ~Bҿ//2+φE`=mmXL_t*qx cDG֟5k _~FxΎɓ'0(zw-a{*+JJm=i2ٱ}פs箩&܂C`sLȳSF5152RD/^_ <ؕ)o}+P=y-.@@Ro)^zld $Zr5\c3?/_/~}/M+_zdϙ3(9/3߁%}=ǖ-[NsRr<#\q?8?9,9IOUJZeu\ep/~J̹駟~ꩧMR馛(O<߉Yf駟~oz衠@(-7dB}huσJ*jW|^xw\}owᱳxѕ[Uk.oaԧnYV<8+y?қ]q[0Q74,|p["&xa!oR*~0CqgwDo;KNZ{|o} [T3'^{\r\z5 p ΒΝMWF _t{,..:ԅ5ҋm?*/f%-NMl1&,n|dH?_t^k+3U19X\M"*+/Ϗe4*LK! qsLFWl 4OTZLvɢ&%xJUKvk} Eт0@U)9%u~VE2]>:D &/pIluvg$/ZE JJ J!\ӎ9m'1xEtT4,$O:uIɓ'^LHÉ #;ۀ1WV^%_b$zR#4ePDwLnN:ԘL0o*S$QvDHa@?駷IYM:@sp[#X6qyěKfw{{yozϣcV#H*]WWM" zA wDuE{Roc-Z㪫bu]wy' V1vP_뮻 {`QښtַJ;N:(}k`/0``(6%}]$r=x.2+Ar 86uYFUTcӦMw{i$䒕id?}Svʡ>xQ'6wnz喽Jw8b[uo^"IW2qO>wig1KGx%cC tL1|c/+Iyk>2/j҃Og5;dA'cl/b:菽XoKT>ET~_7y~!d?B͟yGE jss&nGS/~ƂiW_zUw0< a+yEg;7 b`fwlyr23 OYXp[2\!ɰΘ>C/ym9ى * txnkmmFxeA 7 ۋlTYd<`S]qX]zgv~A,CfQbt<\5Rv>|:[nQ Cr4.0ES\<SR9"Vh6GDwcUl,#b-1 R8k$2Zw CJb\U~Q.<|V; JˊJk&+$VSi[%Ne8`P݄ک&O2j 0!}e<"WY3TYUFў[+*Q`yx\CC](kdtˁ?yjGdi~i*ŲHO~sO=/0)U=WVx/mb7NPS[[Z\bKzXDdp)i(wDk?'5>Hg?,c? ʆnN1͒n&=Ɂs`Q3[V@dG%g*,ya QO<>|W_vZU CA>fV8w8ǥzJWMSe(!VHenBtDvb LfeOla9xٺzWgiScy 4Pfsl`.#g JVdSC̋o> 2︋GOaS|,4J11O*3سvAk]]]={6 ' !Bdh~y٩v|KJ]hBkEI$[8VH@<'=Tߗ 5a4Ň(L.5P-(*MW7=-3JJAҊJV&4zxh]3X8n]r0Zح{2zQhQG ~ J!W1R3v O$g_=U ѪҢ+N_(fejyHq@v"P|3-:1W?2bF3?KȦB4{Kw/kIC4wA_͛7s^6ច5_JwW #Os=ݸ2914Kt ,WnmnVWiڵ:GF!S͚[Kz3(mMm;Hv/l44O_ֲQ33̴ه`ay7mX|Ĵ3jٯ;BFP䕕ElQ*Uwoyx Mϑ+LbZ6>nӛv.}ijEP|<߿3=X{K;;ڰ {o k`-'vƎ{;'ϟ mvlfҒ4h+R*_L&&T1c aY`R4<_6g*_}'dĪK-\vEN/O@P q@A+a "[h@*%!+2]E5yE`X),'jĲX1+BFcgÒU-:+jNY4oeՐ/;ttu)!&rC@4T\( Vc4f;O}eaW1[dލQF֫iX&̴V%4V`Z)ˋ%^KƋ/?_c^[[##糖j?O6lpZ>f 7j_YY=7)a{:`/=[h& 7_zpbeqI_+ȭ[7ڶ֡j}C&ww|Nxg]hnJ77Bܔ Y?Ƕ}-Yr6܉f2-0Y㈑wpfO:,ƣfm!}j kAv "?n^?_SJ V TK}o{Ǎƿ~OפsyiEo{:Y !O~['W.=8ÁK=e."8 MC܀/l}ε_}ǯ>ۿ%LB/6.iI2%N=s]#+rFc@py9?q`O/8sbG-IC~9X&$9+ vE(#eŹ ܘкc̳Vdvb( (4F(L̚Ihn͐y_ϸksS ]Y4p=BONi/䛶~98C\6 [ -]T]=j\C8- $֠ CaA|Onb*l wVz»PɢbH/ V͐^#V5ft[oxL6tgRvc9v| &N"'+ƪ\p.&?!끗Ќ_yiʦA]+.;8PX( lܲ]|ZaZuTɴsS"bz;?;B/ؑ&jfrE#BF|}OSr+j9~W\qť^4,Hiӓ? ivI'xL+*Ncz!+RiZ^w1/|G[)xYF#j9SWS7}5%}W1C%5qle ș_~\z/:h1雇WlSfO(]zN9O5%[Wlnx腗s8xMg]e%H,4K}zM%8<|هiR!{m[9y8a(UgsDhT<ۀf|!ys/ܪF%XdYP`- @|@]bW+TIc gɖy/O RR xm0pcc3Ը9G|FdCa`Z0B& P0*4 1hpl=Pi܉KHMͳBH8 r"2I !b@*&k~q_kyAc% Uƕ{{?,+]n[QEį+6b1JKx/3 xRJ,QW>+^,@-+ #Q%^RcL+uq`I"w;esh)35r#ο16%MsWfTW 7?.g thr?}C+k^0yeZPx9Y>r(-1$;I:qFN_8]YCrˇW$=i2YfQ)ls`s^w2whUӋ:dgKf=s4s8'pPwm˯9E4Q:_BC柟|L{ϻ3?p"tnoT景Ir֑S/>aP_R/Yg(FSq7~ނP2xT"ņG̔L'g{k$c N~fjd(:dG&οni~HVfsYD_E '[`BQsjAQ d;qWȐ[qb8FdE<r@:λ߽tTI5=z]5P2>TV/ BuWqg_Z=t_y44z~owXLTV^xA]^/l.ވ|M2^-NFdJc竕nƫ$岢X?A6sXqB3"{ 3A5#WMPh]d+HnmlJ"^ާPBH^u1=r>)-dv&KfU܅~ck@2ee/7m]7y[N%1^=إ>ÉrG_ʨ 'g ٷQ@f\Wt3Ý< RAljd~GCFvo>Ͼe.;+N.()OĆYCWx vXIf꜅X5Lϝך2tNm͋pƪ6A A72-`([6VzҘNv҅g>EI/{4J@UaC\e< u͛Zsw[\YXFIVl1֤p ~p`x.?_%f`/|O(n1"kFl4*<JdTX&+8-*[.bIDDuYFL}]j\a[%_KUg;b9g]wxC??G8Zcۉ¿=_x1l]bg3gxK.Y=42P1 o>hhǞ&YS߽k]vmWp7o_^[J_C0y̌x*-e%}./oƭ+5k䢏}?_s~.90b{/y߸`xdKptќz^o;abx-Fp,]g^.8fFUG{ko>sgog??aka8Ջ},9xm.zWw_S3?Sfa4+u]], N 0*}81O/'ڂG`h[Pqo([FT/2e '7-*D=%9W0b-af)x PENcM xHCb&d= 37 e`5|9d )]̦mM#{uhpCiRn9(f]&CEu' ֙pP"%q'Z`"lk&nZ,,h[bUN&Cǉ2U4j&n,{@C%V!{ R` %[eI>"sg .pYJ/!f$ &fܡ.s_&Xý^N{d.IuZe!Vt4ogtEr3(KĔ#9Y/$3Z>(/7,⻙- 1UD@i]>X ʯFK?yow\O+( Z^p)_pGyQgi⋏9E_hθh9o=<;37u g91zhCϭ?vEǝsٱg_λr~E=咫z'o~[7GzqVB:{{[o8C݁YJWWFzF|+Ru^`Y/܀wO&?Y~>kL%_-g-^_OfQk6WO֪%=L3 WWU=t뢅Kimљ_s_,J ~~hBN >?s(.b"}EDJ{**-/ 'U<_Y^5<f[h[/4.Oˊ1?kjCO`ܸ":jƚ@!7h1)@'@+~Ah@K={wN{M2{Ϟ)S]N>wL1uS'֭~~|^rݹcS;wf5\z12Qb(i|DJ/y}UTwQ)J7.Xi!տl3WYv?ö́" /orY'ˑIA eEac]Xp}.ji2u;^3X&8C'o~uvwmBa+a0jf媕|=0(Yu05\8{1[e2~e0 ށuw,=I{6RRI\1Z5aJl*iSAԅ) -K 5:ت ˴jp;|H<7BK(DaX]lRNL 2uνDɈ>#lw9[HT@'skXy/ۿ3s&'[We]?cH~;z5Q6ኝ $EAe,XT|*e( WB3eQ(EQxo{ys&Od D @r]Y;Le^7p$܅1}DTA~AY"s~⼓Ni _ gmk~X233FMO>ޫ=벵x)m+i5e$n^ MUWPC֝7̉7goS\*ͨ#HѓAF9J1SBY6!.LKn&s }< @Y; ꓅;ޕZ.V6%IhƄ߫ɓH@TR9zÓ,i4X SaELcinK>'%[IZ8*~SQhO,*zdve9OH8!n4+*,G9LAсb5l!ۘJ(tEu788zaۡH$*#}}DR@Ual%F֘,RD學ÚJwwu_<4 |HhZ +@4!xGM q>84X?qbk[@-@CiڷN<ڹ{ޡVa=pm#j8Ki-ٺe2/سg/gsccSl8>~ROw/aֱcF)ffCK`"8G|FPtI%MxoDt,_l[@f%j(os52dTpp+I LQ|NG])vI3V%:IӉJ-iӀ{yގhIhpVF*]U=-ZPDKʪj'%->ڴ Lƃ¨0F_/e(t4`j aEyF*DL05M2!X[Kq%aF0>wcW%Z{e>/q, }cוzt\²)@]x%.U6!U:1X6XRy{zWS_ݺQ)@Uy ~~ `LXUg0^6C{DDb7h\`c%S _9)TUF4TlJ {d25td2fC*1=|Bџ/ʊ4RI\uvP]MvP{Ɗ|xg8r?[F{]Uư+k,r⿒H+ȰëLV}zUN,ぼ4M4 XH-\zi)hWK@E0XʹGbPI,A2ٺڑ@S^b $l&IFFe+CX1w?Knn%8 Cu|v,<("%O"ˮYS_ E2 [ģKG<]/fD-ҽ>Z;s!嗀P1 ꡴l,0-qyu n#^T!9PZCXK( hpO`Ciiqu5qƗhP݃d-95ÏW$=gf{G+?7PT\}8P7bi=}tX?i} {M2ӧ@\c*2')v(f1lxl 3X+$"YeBYtY~r0\3c9С;̙Cw:(g2$n)F|0q;l]UG,_k&+, 畖b10mֱS//#*Q5vLz ˧L=bX0YYXTw~$tK LLb|q%\lf9zҔÈc!)tLZuڨrjNgŘ)No<`'>ŕ1p1!OJ07]S~&s]G+[8d'>3Pd!%b_`T@E]:_m26z%Kif?_~y'g *쾙W&d(cg]ݍ,fĘW'{{ %lALhҰ[4TfL&C&ozw_SExQ"*>dulXHL7]n~ hF5N0#Vlڞ4jp?K28*fnZ~MR#QBYRяKDCIvMl OXufUPw%gˠgU=EY3ʳQURCGVQW,zVHbny?!ԝ'ŕ-{62&Ei=H d4/EVu4S*`o»t="Q,'Pu*Qf/~[8kF;\/ {/8Z<`8GE7ꇃ+4{Nq6Xc{,*W81En@_o&.ڬ%ۚt1M HثGen:Ď.,5Q;类74klh:e:k[P?Ygpsv\0sL69̖lJLؾ4h\ll`8`? 0bbYD˛]A4>2X&KPwBd)(-W3~+Dd{EED̽]@B얈X~iiqĐg!yDTyx܋3H'Ҫ Pt_*I H$[,^[R^~0bVP#oB;<,A9۹VqE06elkc: x>af* ';g1Ck[ yS=z |&_Qm~tSY0b}`IxM:gnǂ '}x̡K^)*5O s_?.K9_AHx<Ơ0Oc)xhDa%$}3E6.H|αo]#/)*imƷ",j\%=gdjcd'i7pG2{h+`OqհgʨdT*RF~Z2AUCMi10a g~KҥR O=dK^Ph8>P3j1&J,wY1 m-^:J8Ii띬0ȇkK ۖH؋Pv:h: EU"O:|PmbW2c 0P" S5|IcX'1Ji~U"v$Q=8fҧ^.s8Z>|Z)X!u~᧿[$B<"ƌ.eŠ2"%fqk䮏H`)ȽΘ2^،/L;wrpSΜK[PG7q+ڔeZ٬KnKkdI#C1QZT~^ @IA+4s(qTbD,(d= g7_SqFX LQ0zD4 ȋ;&Rw?טRsN-*,oQGAB7"El^HV^1ϱ& <@2<<=:Ŋo2ou"iSd xmNLcriot"r%A:=y_R!sȊJ[ :q2$2F:c 5sȒ2_ZP5/[cۑ:6+A( aAbî':C eÚMU98 -9e˦]kϯ7yQk\;erMt9c Yymx͋Y?n;n Z<-E[zƍRBU:aٰf7L*dz75zh "EU*loyV$E]u<HL+,*mgQ}YgVL'SoLPjDus5G%,I!d([֗~Zx@kMW;ÿ{WM&RГjsQ %KݿT_?s~0x!jGY|Wgӌw 2كk~hUeH7j號:~QH`fr2XbCr:xIuH+J"{$=4 Re2,#ChUUćRăut \oTGt%[1}VGiY;}t/~OݦGkļ/z2 4ҫ4ŞyM6E{t6֚qC7i4k^"^mF>u+2pBޚ:q@*0z =.c܍z.v}EcXˮL^xkj2EV@ב荴ATxq$,'; &?>g\p :@bBF(&BxxgP,CTN2E'MA9õ6W^I`#dP^$9x3 Ф1 f6Y L= uw'R+(E{k?,H)={O 6i-ٿp5j$@`ڲUe<9H/Mb4 '{GWMc_7u&ff44WlڒQ();&q0!Мm*gqUi :cJIFݥQɍ`::Z)؎v+b4R}HYccsz,3f@TƱ-Ws!kW^lЬg|KKTaL }2)Hr9ddnn頀dgglھh3 At̯:}'ZUt/b&o=D$eH[|ctA_|e~x38u3Ǟ|ȴӏ}Q3 pՊm4;Lfgd0IŃmhOV4^9C jᝦ7h!,h-ndI%P;yKwg[FV~fvV6b?16@L" :^rgud(,[TTh3ۢd2cx k|y*]'߸jXi݉S$϶pKu:к H cwr5n'5#v(mGr=7b77cZVnyk;*rW.Qo:mΆZIA5p_O#1̘3ޣ :X&~6kQ8#XY*OE|V<-4OQ3N x@U(KW:\ME5"RҵI"u=H)L+t2#Ax$%)=F֜(aKA`_,N Qy'x1C q/gJ$lFۤ;@7/;E^O"H )Uэ&)Uj;\W䉮W،œLv.i*nzd*Ip"M6m+M".T&͓eL4*0Ii5/ik @ Og>F=IA.;yZ]]csiI*5ҲuV{iO]LAE#~t奶qEKqR1aЀ!Z9 9t8hh-H>ISN#эQS:ːhdhrԤaFЯAt yz[nN!(Nb5eM_?|ʲ5 ǎq̯+V´qcGW/ڴaɸsmV3}=[[9D#̩ڲ8?}-w,,--IK#)ŧcJʚln'jA~1Ve/ ݡhZʧBuRVGE?(8~ʖ溢GO&)jz4ƼU-EjeVEرb$nͅQV>0#3uMYYË &N6,&(`;3>vUI=6Xɍ1?˿CH@7GM|OtaAѡ~y`c/|CVagԥ-U<ČM.T5M5kҫ`fJ肼#)[.?L.pJF %/^Q,((FLx?^yݳ}'/}N6ccP֪|eт7kJHu!YN~FTa?xlʖO;Bpu&IRU SW|̣3!Y=wN킷^yV\ YއP'9h8eI5)?25xs(J)c.bG]m-vv9=iNEmŅ"T!!~uË@{豐i 4zRPh1r'{z m"t=8E3$:t3&uMlP~.gϯ玟4N4HD,&;)9cs8訌BMsr2s(2jjY zD*%jh+(Om:b^Ykdl.oi*Y(\Lܫ^ $]a<q N$5dxjߢ$@VZzv yEѳr#bFegN}O1j egjCG-%s`xkwCփ[p 2}H|nuZ2Tn/1ҩûx>EBei10{b?=k:m0MV K`/ qAFtJ%|w{Ś]qڏYJ轗$M^qXo%E~Su*ؤ&Ͼdӈ&g]y9w7kJZPFƯPFn8cNrIg?%ZT>1b}5^ޒp.>z_\u1RYe*UmVOfA(E/z +\#Y G/Z3)e1=}'3y~_ۯS@KQ]q7mz_sW-V?)"(M.̄\DnM+$Mڀ>$ uqbFʍ$&S {H8 lk6GY]d(nW♫J(OkV._p13/>xNYlюLŢ tJX`̙WW7'͕Ku%:HtdV^|O:{˾U $&a#M١#ng["*BJqzVlPygIut$G*RV_y㳐A]n@rW$pKj);9':F@ĭ uoO_i&k=P?H!~(}Ηm6@UrrÆ hD$B"G;''w˖^]xRe4cAFנ4N+i(-!&r3[Yڼ@3-ӑ55m${g'S'> QTviYI_`(ٹ9E#Gb.UVbb)V>óƔNه~iئf{ĸٸee$sDlMĆ0dl]lŔMNfSy:Zq,Є@ӆHuNEP|\c (1Pz&PcqgR(<*)iMMgJSKDihT4~!S^9>nN CtKHFBY7Irʇ3_>O9=ȓsHUmSCës퍢"RR_u!*&g63du4KɕOմ`za()#.1ss/hԘtjz_*w}Nzyٚ ash*9pW8WʦU)[CC-S0~GirSn}~knaw<ҳ+ĺ,Wv+k%"EeJ6zY0rV5pT&tD]i1 iPCEK[[C$tiO?WV?4)7.tݢKE.@BL>7^|V]Mu4>qdQ ub)RpvՄҩ\xSG.Y#c!OKbi+V Mh-Sێ߭ޫ Fy_;}tz乧z)}=IW^v2ḃJ7\yїYYUM~O?p]U>h,MӕRYMQ2f,tIS1m\w }i_<77^ViﯻS>~텧=o~EP_ eO=+/9CJ//2}_XS>?븃&~CU7|;vҡQw'5q%yVMW#.>+|~qҳtA̐-|ou^q4z廬W_ ϖ'b4킓:}nrM-Yѹbr^F -P0ddrSrs?tUGkٯW-fןpN?6KO#GgfVTW\pQ|~hMks2G憫JgPVLE[o.:ӿ0&_q9` J&ݷLؚ/kriY]`+]yι_<q'TW vH2]s6L__C`");#PhMq^l{d@D.6qPe!1́9&][l8$d$bcj5Muô',f/tT 5(&u[AX^h5gw#dQ wSG"=R*68C(hNvԩShTA!C2ss fڂO}{kS[K˘qc۶oւ@S$ds@JuV!A>8Oŭ/Ly9儥R"Yܳ6C*K,'OJJ"=ZI@][nzvo0l@FCSfpd4:z؇E}5rwGK|j){5C12%eZ]zZ6W()/LǄE-[/&b֤ᢠ߬/Uڃ~Wڿ#/?8Y%ͨg5kU#Iɤ'Eh'Ro|sѓ]&g).>,bLqo~@c^wN<{~¤iů-]pob<TՍVI)ޯ@%/*K^u+O359cE]prQo^:cƁiYUTL'~plO<(Fn}sxͯ붗ob}Ჟvb=?滎W~E)@B+%W^z-7t%oj6YR<:kA0˧]xMaVUn|xq&N}㕹 urG}݅oj UU157{lJGw"84||ɔP/+-C͏qkVg~pM[6W?GyӺֆ:#--g7o0qfkW.4|ȧ_v?k>BT"AVÌ xNc0_z,aq57eƝ6YdO}¦t]v~ޖ٣ֲ@bxhy2qMA\+1+z;'jGJdW^NQ~ffp )%)&@σ1و(OE >(j;3ReLj6_k˨- w(@{c ;j-偸HY))F։cJ@70uR۶UE$WB7Joi9䵏˫۶f+8z[ʭyhݼuXP"YGTlnmlknl"{o[r)X)ݣ ۶{R (SHQ:<&^XDV'#wg0}>{ӌlHF.2S%V|yb{dz[b!Gkc?U`yOSe |wOևo?6q&oZfwp/dYyO=tq'e`nihzW6+{:/q,n %St1?7XEcPv|_2Mub%nCE?e;o\Ӵp/Q<ꋧG'z!uD41]oy۲rW1kb=sFXÍW,=sV/[S /¦] zD#8=93s⤽Qɤd _tu 5 D*.7a*60Sm8yǟrvtχ?jيzh^|v͊UτqMop|+/ 3}|N$/mF!7Ԍx~ % TJ ӊiqD/-[q.bQ>G1Q|IpK]ϟp =0lĨ5+/~gE#0ijps_ǙٳǯKrL!/~Պ^ARY?5kДYs:1=p,bO\x!~YQeeO=ɵPV-^fKcC[:yC?wuwq)xAlۼGvq'o)b0Ξ{ٳKV#Ix<zI{~νgii9R/r,rp:{ cE:YyHIDATv85{?vH^r^DSUЇ[[9tWN57aiD]ĳq^2m~7{풫GkI`+ot0Lr%o9s_8b5^s 2@P݉V/s-^4sUFT8gK'sյl)W,Qq+[Q<7j^3RkV3V/?7t"?+|ukiu%Ё!^\ݝ,zCP:i?~S^x_p֊pܖʍ6t&iǡfǎ}m0NʶޓCOWQb+Pca}tViabE'(Ҽד.?WYzH1l c G(0q $w=r]xu9Mn¤=$.{lZkʖ❅vbɞ憽g3c(oZ[2 b]$:S4EnTm DNM#]NWE(Ī8$jdT\(>a\dbӚrY\pܤI`%`s5izۋupڌ9'Mc\{7pI<ڋUɲ.=|4K'vRRO,[6g&@OxɂYժwIRQYЂe2L `N42 \*wq0aL;V:!TB<-gaՙ֯\FgE,&@ll6gdλ9EbEo1e&k㑏Gޑgc}I '{d_D9Akŕ2]xm*JYz:4$4dsrG#Q-sbQlAznANJ/*.C]%2ڱ"1$%7j1n3]8 dX*#̎겓]r)@ &[nPŬ겓F]֡Lb&hǏ:ztcX;;;[qSG[Qd>|X%<#~տV弆Gj^%&jBmjԪ_W'nOQwFS)DK,9VdѨ)=xJ>1ՒO(/y$!mR@T>;c:|R]cKO-h~Hho ~儞>ZUc%xFd;o!|O@F^#g@FE8ˏ%<uˊe,x_z={|mjpk'0-hnVUmH|kM(6yϙ6s645,_zQԃPȱwgjjZMuՃ9?_Wy%G5sk5Dd}2R_ĺjj}wyÏ7W^x#}[֫/>NlڀYoK{m71e＼◞tUY:;-f9{ix֣S_u)9?p׆ SLwmX:59#IRFx߶vvm,{KJI2eݽiz"=x^|#椽f$h3JX4a"{5lk喊7?yY)9v?P~MzfV]g̯]tȑ>샯0u{䮍$2ޘ҅on{{|3MΕ;S>|ko{(ѫV7e|- =ՕU/[b 8-\bc}]fӏzh{ԙ,3}Ujog>w+|yiKMJN~q۫*qR];q2a{N2?$?bQyKdFIfq@3 #Խ=D,j0D/:TRidyAș>|TD(x )4=H?,ۯV1c 많^;ףrXA=x(.rt)sl>k_p5b<VQ9O"\8 F% Fˏo|p\D@X`F5}q >$-pEOH pXZNw2^n@UkSO= aGJ!/I#/V-mj % m8aliڲe+4y~=Н(>)C@_8b `PhCnbH[6<6kYa Ly1'g +,G[(ujvy2r.bEF!=GT)~eUT_ĸ]vfZ+ m+X#Ҍ_K*H+>)R]he/di/Ҳẻ@4 ~Q_{39 <{Q(m3z@< C6M-$[M(38ϔy3nL͑<ȁd6ټ(1.P*9&/(1[ن ]!ޠzW[~DPK jaD*l TPB&U QG^gf)OQ1N)<@-I:v܈[㚆Үjskkɥ+'N^f ;Ҏ*GՑES(H]H-K/o6R(ۀP-{nӾ=ˀe$7% /]dmIsMQkhd+O?T4elZG1IfyoFdX SއT)Y{6`:hK (?R i 靸@n*$YQ ݁6ɌeRG7\)ChT topHJwrؑ5d0+,qS4\i_RG< >OlZFRИB #FY%1J:h!^m;%Rܸ\IC+bj!Q݃+Y*%ɪ&.bz;VKé]P,bv*)HR|nU ZH`W$LSR&S8W$b=rwz9QQAR4ÛSk XdtCAd[哠;gSg$S`[x&5NlgXFܭLHp 6'9i"'_rb_͖R2I\-TJbVQ\dB3:{IHl8.VІcXe Vw3qcNMңGyP/lb&FG[$4&3)&36;j7(]WM]-i( \C݈lziDkjqra ?s(tlT^fWTB [ƍp,#*1Ra>C̮VDi> UHծlj,+P+ee! m[î;[jn`Gg~sJmڒ^ RQR2iP")ǒV.#w=#Bk浓;Xz|8MDJD`a2QPJ=Pg$˚Ϋf+7%&0)Ri.["$!"5F a ; )(4УH}J `H]0ErҴ4ꒇE*\=Щhƞj/D?H(E0<ʭ0g/mAX|60Q<[rԑf+"!>E%ɳ]ОWK +,pNJ cR: I\#wXxTT"1pҪJ܃r/'|\'?A8ѩ?kTS`b(,n{v=Ulńh%呱Nq/K% fjkѺl %NTz8pN.!e6A[~'b'FAAoZZw'2_HI'atv옋Ob<+{/*$k}) ٶRrwP(ޡ.qލωpOy0@;i89B/ըWÂ RQ]̃` 8Dw"Kr鞅3 uFup:BL*zDaXQ/n!G}Z>'K0WXhM 'u&7TPh9 jAe^zLx+c1!d~]9Y9۟9D ֏T*y# &$j =ld:Y7$0v؍-F位Bz`e!6Y7.pd`#GEb(֬YC-ۑ$j"է)TPسx:vc1RKSks9]b6 *MeCaY'&HM:1yy0MMe ɰ]7/z.xԝMjESLj4_Wdd,"rlgP.Qx;ڜҩn\Ek4M@È .98+'g%E\ۚ5k931Ӻ}{5{ Y ),Ư]}]={9>w@}4NEX}g{-5ёiFCԄcGZO89%Wy摏<&K#Q;>]?+%)c}v{$|L"z!A'Nn'UA'`F',v\-G D~UF'M 0\v۩ 3+cZ`Q;٠N0vyrW=&;ƅﳮ}bq74&<٠,~ "YJN6@Fl~#G`1qR(6/{#_d3iD^&e'E(ǌ/.IWa ѦtG/~"G,xU 1Zzp'VfH)T!J: a16d! 9{SS/1LzHCTF(ū7܌K0tJ5zm=WffZCF1M^ 01%=2_tjB~F9JR/wYI4HיۣqFAG'Z'eN-DZĳ>d#Y_;>}x0-?H7S^~5b{/)#OBUZ"<+H& eA|0KZ"ġЁC ם>]:OI@P(4#d $ ,H菿mQrAEou/hƵ'c QǎDˢ}GN~곗O6ܸĩQϓy!ZpR!Al_Jϖۏ!W"<=d:$ njUDP)wQ(wqxdd" {:mA/K%E/rkȣ`q$H;vLiiɸqcHz2sWaې&'isK&mF&P;ʯ]qr%`bh.dQPqEϸ_-vЊᴄ9ҌbbnaLR'<<5.ûFn.3!PtA<:%VMLfw19&ԵfA*)ΫiOYV僚kkvutcqxǷN@Q;\qahtR}W"S@~nl۲U Bhe1/AJ$pʱb<%$Ӌ`/JDdP6gGtη/l }$2IlAц qς5LK:@?A'18fL7p@,|3#JCoHD&ǎ%aw9jp:n@ (\ӌWA?!!s T ެi(] ^qsL)g*"0'A.GA(ӈ@:[m,ɅUFOs5Y\`V6&ߒ\@+Z2=eDZ~AVeFiuA$Pn9Zpmޒg5Nܺ}[{g&H,u np^ 10V;d!nv%4g3L/TN 1;t_;/]|yygC=VZKB%vo~ZA|$elh8 tK Y{M9P)6Nf 0~x֭|BbĽk)9H Dhy<9W4E]C>_1G_],>KQM985i0hQlԒzDSGɡ?J']#3h_NX0SD婎v)|/z3(Az#̢$ڑNmYț5¯@KdQzCxPc:-i6PwCpHU*9:lsgՙ^bhMKlhp%t.XaBDG; *B#b^Xshl3otڤ2aD]|uXǆl : o`-.qIAL(tTKk[.ZrR>3<|B`gG?/ƿ7ZǷ)`M$A 5c^ݔK^ػMU XU Røu:(owȊwGw!v Ci 2"f!`[F>IjdYn^*J@0S/}g,~H.HNMICP2NNH]PmMj=ycۗ S^ 68RzMC anLA[ K7yH6L(NKMe&%&pon#j*6653z^NNFS6nۉ{}FsBZm5;m.Ŭkʬӧ1y¸1EÇQ"9Ir 3궎{IټWV%`$"zo:KW1ܴ9XqQ[bE'j݃":yyctˋWQD<|n 7(^xLZ}nyÜ>.zoҾL$SSd@aG[yM3\*;:zLN%zw7}?:c5 _ wDЗ!tkzav7Jbd4:"'˛`#,טH=(DӦA=ә{t`1xt[+dM>Li0Ř7&_Mh8B*(L "Yr] vоqL)UQHckf5h) b,yǛ@uTix4\on-&O/6\wZ;!#=NQ2z>E3e~wJr8qcl"d=}N>f\֘iS[WSz*41=%2y Qn^nxo]'N^pg`/j//X"$4de]V\0wk[=93)>4H$obdg Ku4Zj {#*U2I>)w{1_o;], 2w55ׄ{U :+@yFt?;wdBuQ.ӝr,5i{~<~MHtNl N^E:)kRІ@JcBlal]0[|wՐA-h>渭UkWb7:1v/Ii*GǱ yjia@Jr}`4nǌ[8jf1&[27#CVZ9eq?RDdi֤~'YROeSs@+>jW>ME/+r;{3Н7T֑ ؋#BZn0lq aؠ霓 l8hyɌKt|P@#(|FtmR?+e0 vIAUir=*9<1<9,YWJ^W^y֕[+Z\1LR^%KjUiZwRPDBm E$˪?UdY*ftt`?uKC̸9_,4HδprN0=//N{OP{eK1bO>0 `W owMwT;gךSpC.'oGmjS!͍F3]&tSzMs=K)Hhx2j{hL*3YťNZ,u`ê4 83ѮQ zY([H'"*g,Grno17T|Z-6@Ƣd+GW%7AE?hAjG s=x`875U)zI$ 34its,=E=A7ӺaKR: S"3i*>QЧ.]pdإ*YԩlRRC|wvuy_h`.̶e&/E{^P ̧xh6U*G͘ >7t}?J A `֥&^R89=NKok-D]q`Q`'hcn`2U60MOؑv>1"Egl^UwL6id]f53[\z0#"hW2[!W^_xV?=ccӯ,nr~o)CQU AP6Qguo E22REGdKDF#PunMInTe"(A%A-bAF! :pP[](+rg妌ͤah yۼ3\ L#6;ʏ$%3YPZN !+d3=+FE]H_2P}5)\?盧O)'EINC,A.4@ٍRoJup1֏ܧW=/IE}pR J/ҫƃ>er`+B`zf>ytGq{ܹ]/]F}S起W)d70.G Y䫿;l!,yD,O'|-4d@07 Ao5 jeVX"E֗quCxE :2:c b "p! OCfbܲ~]A'ċ7]0cF@]ssA\rUQ$X PT̠;ˀ53(n=N?%.4~E;%̀kܳ@<;>.v0mDD #m P4iSi?4cߣ4PT#ARǘ[k[Ժ%I4t0L,uΖ9xܑȽiG;&t7lkhji9ROujK | K%:(!z֠ gPqxJ+S&<RcN94VJ=jfVKK%{9/ a<$ؼ 4+pEH LMlȳϿpAY,Fިhz!qШ'\~>5ܮ7p8 ?~ *H'^{@]>gFmQLq' P#gtgyec5zG" fSKhxIʓ*5(OJƯd"_"56xt*#X:_)o3a;#NH15enUx_@1IS222$Tb JmIk@-,Fiu݉0`b %fPu]0EbᢤY2V3"hM'"(x{e-=怿.[bCigneΨ.1a[D]cE-WYLP-:ZkŞ%B}ޭﯭּ^MuֺSƿ3b5 Ojy{_ZucnZ\YT}ԏ=躄p@vtvK>aQm7vMͪ)0Iyյȓi8>Ck8@K͗K%wqE5q|??5תMc`@GG}Ӧ[h`C nD}.?җݍ$eIb|ON{^[~<.pqB4]n8_M{FX!fHӆ58K-<,{'47ډHCՌmZ~$eӑHޜq9w!|+Kqo^S_'NV,ҡEWYTQ4Q qZJIq9HmY.(oO96F@SRZĊcp@뗋=AEc7)A8U{wZ)"j~xEYY$65Ñz) Z:{ -2)KM`֬T$XKN96.=iRVj? zdjS,sHϓ4md_=;]2nhĶh[^m+Y(⯦pK{8JHc,oG&Ɂ+#s{^psB~Ić&5-LS+&W I+wXm[߰13t>]?.xneͮw^~5E v8LԈ"pPF'PH3fv۬m7ru+vFfoa +5Ia'>ʐӦe+pޮSsl#Ko9xm!;}md!1-FvC0)mTPQ$FL@?Eb$gz&>3M^*.ۅGU0,*R:uѺfMu! ȈR0j˩`kq/jϡ|Qp~s:j.L'ji;<{EKhln+7G1*!GfHz̓*nAIg|}[+\iن붶6ԓ,RTJ;r L&VRI ݣrSKG?ȇJfO+υeD:A+=+E464'[CxWc5MᶆCv叾 iB ;u3k؞Ďz,e4 !5Rsd'xy1⺞E"!cj]:Hf/i̿NZgnX55`3a(JE?Trg,WZ.FaTa ӾJ_ y/7a@Qz}#?RΠ4'3nz-T־Vu`?xT=@K~%9?|H[zHJ׀F%RCof(,Q 226UO&z{`x0- Z곯~j23IU`x8t8In;'Nz;D=Idk J$]վÔB4% ?18^!1X`'+sQ/&͆k~v`r~;7i\=)"Fb8`aSn堘ޱh۸qSKBs2복.qN~{ioQ'̭X-GC/`/|֎X,ホ\TVq` (b&W^$uٯC+:ā!|900OEm0NHny$9&⤩gd3ؖubOX5/f]xyR8F,bD0G.x?LZ^萘~@[[9S|Z|LLèp8(栏%FfvtP, 8N:Ӟa@1˪92tKR`QVzqɿʷNY~O됺OdD?p{-WQmax CWq`o)&Ύ@X#?( VJC(4!3N/g:{)'"gSW.߈C-`M+I*^A11!αz{w$7,*HC(YBe\y\5I?pPM/FH%w-yYXD#ߓbޢ}-%z\ShaIezͯmu`< ;@0̉z膀Cq`l G6ymֻ~Z2?+?&dIǁX@Ix90sAa-tpna]_iD?7{IENDB`XP( L/0LDTimes New RomanPPs:ؖO0ؖ@. @n?" dd@ @@``|DR1% ' ! ("#$%&'(3)*.?$$b$cjs^bd;4I0AA@uʚ;2Nʚ;g4JdJdD:O0zppp@<4!d!dv0Pls:<4ddddv0Pls:0___PPT10 ___PPT9d\E?~- j2009 Stata Conference: Subpop Analysis of Survey DataO =HEUsing Stata for Subpopulation Analysis of Complex Sample Survey DatajBrady T. West PhD Student Michigan Program in Survey Methodology July 30, 2009 2009 Stata Conference(kA)Presentation OutlineIntroduction: Subclass Analysis Issues Kish s Taxonomy of Subclasses Two Alternative Approaches to Inference Variance Estimation and Methods for Singletons Examples using NHANES and NHAMCS Data Suggestions for Practice Directions for Future Research" Subclass Analysis IssuesAnalysts of large, complex sample survey data sets are often interested in making inferences about subpopulations of the original population that the sample was selected from (e.g., Caucasian Females) These subpopulations are referred to interchangeably in various literatures as subgroups, subclasses, subpopulations, domains, and subdomains, leading to confusion among analysts of survey datajZB ]L 5)&@Subclass Analysis Issues, cont dJSoftware procedures for analysis of complex sample survey data are becoming more powerful, flexible, and widely available, offering analysts several options Analysts need to be careful when analyzing subclasses, and be aware of the alternative approaches to subclass analysis that are possible and their implications for inference :Kish s Taxonomy of SubclassesDesign Domains: Restricted to specific strata according to the complex sample design (usually geographically, e.g., Texas) Cross-Classes: Broadly distributed (in theory) across the strata and primary sampling units defining a complex sample (e.g., African-Americans over age 50) Mixed Classes: Disproportionately distributed across the complex sample design (e.g., Hispanics in a sample including Los Angeles as a stratum) See Kish (1987), Statistical Design for ResearchZN r *(-Design DomainsX = Sample Element in Subclass..+) Cross-Classes-* Mixed Classes,'0Applying Kish s TaxonomyThe type of subclass is critical for determining an appropriate analysis approach Two possible approaches to inference motivated by the taxonomy: 1. Unconditional approach (cross-classes, mixed classes) 2. Conditional approach (design domains) pZeZZv-.+The Unconditional ApproachQAppropriate for Cross-Classes, and in some cases Mixed Classes; the subclass of interest theoretically can appear in all design strata and primary sampling units (PSUs) KEY POINT: Allow the software to process the entire survey data set, and recognize all possible design strata and PSUs; DO NOT delete sample cases not in the subclass! R k #D*,t3/,The Unconditional ApproachRationale: estimated variances for sample estimates of subclass parameters (based on within-stratum variance between PSUs) need to reflect sample-to-sample variability based on the full complex design In other words, if a particular subclass does not appear in a PSU in any given sample (although in theory it could have), that PSU should contribute 0 to variance estimates, rather than be ignored completely!VZ !u!=:The Unconditional ApproachFurther, the subclass sample size in each stratum is going to be a random variable, and theoretical sample-to-sample variance in realizations of this random variable should be incorporated into any variance estimation procedures4C0-The Unconditional ApproachIf cross-classes (or in some cases mixed classes) are being analyzed, and PSUs where the subclass does not appear (by random chance) are deleted, problems arise Some strata may appear to have only one PSU by design (preventing variance estimation unless an ad hoc approach is used) Entire design strata may be dropped, impacting variance estimates and calculations of degrees of freedomrZ YJ6EB/The Unconditional Approach: General Stata Code00(Asvy, subpop(indicator): command varlist, options indicator = an indicator variable for the subpop or an if condition, e.g., if male == 1 svy: mean, over(groupvar) svy: prop, over(groupvar) Stata drops strata* with no subpopulation observations from degrees of freedom calculations * Exercise: repeat 10 times really fastf) (4( 1.The Conditional ApproachAppropriate for Design Domains, where a subclass cannot appear outside of specific design strata The rationale behind the unconditional approach no longer applies Certain design strata should not contribute to variance estimation or calculation of degrees of freedom @ZI2/The Conditional ApproachKRestrict the analysis to only those design strata where the subclass of interest exists Variance estimates reflecting sample-to-sample variability should only be based on those design strata where the subclass can appear (unlike the unconditional approach) Subclass sample sizes in design domains are assumed to be fixed, by designLLZVFC-The Conditional Approach: General Stata Code..(svy: command varlist if (condition), options (condition) might be male == 1, or a more complex combination of conditions (e.g., male == 1 & age >= 50 & age <= 90)T j$ GDVariance Estimation Methods%All of these issues are only relevant when using Taylor Series Linearization, which is a default for variance estimation in Stata Conditional analyses are OK to perform when using replication methods, such as Balanced Repeated Replication or Jackknife Repeated Replication (Rust and Rao, 1996)HEfAd-hoc Fixes for Singleton Clusters in Stata 10.144(JStata 10.1 provides users with four ad-hoc fixes for the problem where strata are identified with only a single ultimate cluster for variance estimation in a subpopulation analysis: Report Missing Standard Errors (not really a fix) Treat Units as Certainty Units, which contribute nothing to the standard error Scale Variance using Certainty Units, which uses the average variance from each stratum with multiple PSUs for each stratum with only a single PSU Center at the Grand Mean, where the variance contribution comes from a deviation from the grand mean instead of the stratum meanP" P1$oi30Example: The NHANES Data+We first consider examples based on the NHANES II data set, collected from a nationally representative multistage probability sample of the U.S. population from 1976-1980 Briefly, a sample of the U.S. population was given medical examinations in an effort to assess the health of the U.S. population41Example NHANES Analysis<Analysis Subclass: African-Americans ages 50 and above (this is a cross-class of the U.S. population, which can theoretically appear in all design strata and PSUs) Analysis Objective: Estimate the mean systolic blood pressure of this subclass and an appropriate standard error See West et al. (2007) for more detailsn=Z1W=7IF4Conditional Approach:Stata Code for NHANES Analysis55(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(centered) Also singleunit(certainty), singleunit(scaled) gen b50subp = (race == 2 & ager >= 50) svy: mean bpsyst if b50subp == 1 PZ22g )MIConditional Approach: Results63Conditional Approach?This approach would not be appropriate for this particular subclass Computed standard errors would generally be biased downward, because additional sources of sample-to-sample variability are ignored when following this approach Same issues apply for analytic models Evidence that the scaled ad-hoc fix may be overly conservative!HMZwADBJG6Unconditional Approach:Stata Code for NHANES Analysis77(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) Note: choice of single unit option does not matter when following this approach! gen b50subp = (race == 2 & ager >= 50) svy, subpop(b50subp): mean bpsyst&2eIz74Unconditional Approach: Results85Unconditional Approach?8This approach would be the appropriate choice for a cross-class such as African-Americans over the age of 50 Inferences are theoretically appropriate Same idea for analytic models Results suggest that the centered and certainty ad-hoc fixes for conditional analyses are reasonable>;Example: The NHAMCS DataAnalysis Subclass: Visits to Emergency Departments (ED) by African-American men ages 60 and above (this is another cross-class of the U.S. population, which can theoretically appear in all NHAMCS design strata and PSUs) Analysis Objective: Estimate the percentage of all ED visits by members of this subclass for dizziness and/or vertigo in 2004 See West et al. (2008) for more details b^R(?<Stata Code for NHAMCS Analyses(svyset cpsum [pweight = patwt], strata(cstratm) singleunit(& ) generate subc = (settype == 3 & sex == 2 & agecat == 5 & race == 2) svy: tabulate dizzyrfv if subc == 1, se ci percent * conditional svy, subpop(subc): tabulate dizzyrfv, se ci percent * unconditional 4Z42 KHNHAMCS Analysis Results@=NHAMCS Analysis Implications!No problems with strata having only a single ultimate cluster: ad-hoc fixes all give the same results Weighted point estimates are identical Substantially fewer design-based degrees of freedom when following the conditional approach; the full complex design will not be reflected in estimation of sample-to-sample variance (many ultimate clusters are lost) Conditional analysis assumes that each sample will be of fixed size n = 397 for variance estimation purposes; no random variance! Conditional analysis results in overly liberal inferences d"Zd:7Suggestions for PracticeConsider Kish s Taxonomy when determining an appropriate subclass analysis approach Utilize the appropriate software options for unconditional analyses when analyzing cross-classes Make sure that the subpopulation indicator does not have any missing values The unconditional analysis approach generally works fine for both cases (when in doubt, use this approach)lPlA>Directions for Future ResearchMore appropriate calculation / estimation of design-based and effective degrees of freedom for sparse subclasses Development of analytic theory for interval estimation when working with small subclasses, which does not rely on asymptotic results B? ReferencesNKish, L. 1987. Statistical Design for Research. New York: Wiley. Rust, K. F., and J. N. K. Rao. 1996. Variance estimation for complex surveys using replication. Statistical Methods in Medical Research 5: 283 310. West, B.T., Berglund, P., and Heeringa, S.G. 2008. A Closer Examination of Subpopulation Analysis of Complex Sample Survey Data. The Stata Journal, 8(3), 1-12. West, B.T., Berglund, P., and Heeringa, S.G. 2007. Alternative Approaches to Subclass Analysis of Complex Sample Survey Data. Proceedings of the 2007 Joint Statistical Meetings. ,(Z(_>[<9Questions / Thank You!For additional questions, comments, or electronic copies of these slides or the papers, please send an email to bwest@umich.edup 0` ` ̙33` 333MMM` ff3333f` f` f` 3>?" dd@,|?" dd@ " @ ` n?" dd@ @@``PR @ ` `p>>f( 6: P: T Click to edit Master title style! ! 0P: : RClick to edit Master text styles Second level Third level Fourth level Fifth level! S 0: ``: >* 0H: ` : @* 0 : ` : @*H 0h ? f Default Design 0zr0 ( 0 0 0` P P* 0 0X R* d 0 c$ ? 0 0,q: 0 RClick to edit Master text styles Second level Third level Fourth level Fifth level! S 0 6Ȱ _P P* 0 6$ _ R* H 00h ? 3380___PPT10.p|zD00( r S,Ap r S` X 0A?P H 0h ? ̙33 0@:( r SX SP: "p`PpH 0h ? f 0P$( r S r S0 H 0h ? f 0`$( r SD r S Pp H 0h ? f 0p$$( $r $ S r $ ST ` H $0h ? f 0 T( r S `0 x 0V #"*P0V <, ? F V > @` <t. ?F V > @` <x> ?F V K5 @` ~ <F ? 6F > @` } <|H ?6 F > @` | <X ?6F K4 @` { <` ? & 6 > @` z <b ?& 6 > @` y <r ?& 6 K3 @` x <z ? & RXXXXXXXXXXXX @` w < ? & P XXXXXXXXXX @` v < ?& K2 @` u < ? O XXXXXXXXX @` t <H ? QXXXXXXXXXXX @` s < ? K1 @` r < ? 0 OPSU 2 @` q < ?0 OPSU 1 @` p < ?0 QStratum @``B 0o ?00ZB s*1 ?ZB s*1 ?ZB s*1 ?& & ZB s*1 ?66ZB s*1 ?F F `B 0o ?VV`B 0o ?0VZB s*1 ?0VZB s*1 ? 0 V`B 0o ?0VH 0h ? fq 0! &( x c$ `0 x 0 &#"*P9 <P ? F RXXXXXXXXXXXX @` < ?F P XXXXXXXXXX @` < ?F K5 @` < ? 6F KXXXXX @` < ?6 F LXXXXXX @` <?6F K4 @` <? 6 O XXXXXXXXX @` <\? 6 QXXXXXXXXXXX @` <?6 K3 @` <p? MXXXXXXX @` <.? JXXXX @` <7? K2 @` <L0? KXXXXX @` <H? RXXXXXXXXXXXX @` <Q? K1 @` <Z? 0 OPSU 2 @` < d?0 OPSU 1 @` <hU?0 QStratum @``B 0o ?00ZB s*1 ?ZB s*1 ?ZB s*1 ?ZB s*1 ?66ZB s*1 ?F F `B 0o ?`B 0o ?0ZB s*1 ?0ZB s*1 ? 0 `B 0o ?0H 0h ? fe 0 ( x c$z`0 ~ 0 ##"*P9 <~? F RXXXXXXXXXXXX @` <}?F TXXXXXXXXXXXXXX @` <?F K5 @` <? 6F > @` <?6 F HXX @` <Л?6F K4 @` <? 6 P XXXXXXXXXX @` <? 6 S XXXXXXXXXXXXX @` <?6 K3 @` <? GX @` <$? > @` <(? K2 @` <? S XXXXXXXXXXXXX @` <x? TXXXXXXXXXXXXXX @` <? K1 @` < ? 0 OPSU 2 @` < ?0 OPSU 1 @` < ?0 QStratum @``B 0o ?00ZB s*1 ?ZB s*1 ?ZB s*1 ?ZB s*1 ?66ZB s*1 ?F F `B 0o ?`B 0o ?0ZB s*1 ?0ZB s*1 ? 0 `B 0o ?0H 0h ? f 0:( r ST$ Sx~ "p`PpH 0h ? f 0$( r S1 P r Sx2 Pp H 0h ? f 0$( r Sh< ` r S@= H 0h ? f$ 0$( r SJ P r SJ H 0h ? f80___PPT10.Fp 0$( r SL r S@P H 0h ? f$ 0 $( r S^ P r Sp H 0h ? f80___PPT10.Э= 0$( r Sq P r Sr H 0h ? f 0$( r S^ P r S H 0h ? f$ 00$( r S P r S H 0h ? f80___PPT10.݂$ 0@$( r Sh P r S H 0h ? f80___PPT10.pAh: 0P :( r S SlP` "P@08XH 0h ? f80___PPT10.S 0$( r S P r S Pp H 0h ? f 0 $( r SL P r S$ H 0h ? f$ 0`$$( $r $ S P r $ S H $0h ? f80___PPT10.pX 0"-4( 4r 4 S8 -4#"&Pp 4 <L ? R 50-29 = 21 @` 4 < ?@ P1.90 @` 4 <| ?x@ N144.09 @` 4 <?x NScaled @` 4 <l ? R 50-29 = 21 @` 4 <?@ L1.62 @` 4 <l?x @ N144.09 @` 4 <X? x Q Certainty @` 4 <+? R 50-29 = 21 @` 4 <-?@ L1.66 @` 4 <=?x@ N144.09 @` 4 <X6?x PCentered @` 4 <N? R 50-29 = 21 @` 4 <PP?@ I. @` 4 <_?x@ N144.09 @` 4 <Xa?x R Missing SE @` 4 <q? W Design DF @` 4 <pj?@ TTSL SE @` 4 <`?x@ W Est. Mean @` 4 <8?x TMethod @``B 4 0o ?ZB 4 s*1 ?ZB 4 s*1 ?ZB 4 s*1 ? ZB 4 s*1 ?`B 4 0o ?`B 4 0o ?ZB 4 s*1 ?xxZB !4 s*1 ?@@ZB "4 s*1 ?`B #4 0o ?H 40h ? f80___PPT10.A 0@$( r Sp r Sdp H 0h ? f$ 0p($( (r ( SP r ( S H (0h ? f80___PPT10.i u} 0-%P#'( x c$ Pp &#"& <l ?jp R 58-29 = 29 @` <` ?@jp L1.66 @` <` ?xj@p N144.09 @` < ?jxp NScaled @` <h ?c j R 58-29 = 29 @` <D ?@c j L1.66 @` < ?xc @j N144.09 @` < ?c xj Q Certainty @` <t ?]c R 58-29 = 29 @` < ?@]c L1.66 @` <, ?x]@c N144.09 @` < ?]xc PCentered @` < ?V] R 58-29 = 29 @` <) ?@V] L1.66 @` <" ?xV@] N144.09 @` <3 ?Vx] R Missing SE @` <C ?PV VDes. DF* @` << ?@PV TTSL SE @` <pV ?xP@V W Est. Mean @` <H_ ?PxV TMethod @`fB 6o?PP`B 01?VV`B 01?]]`B 01?c c `B 01?jjfB 6o?ppfB 6o?Pp`B 01?xPxp`B ! 01?@P@p`B " 01?PpfB # 6o?Pp ' 0\cp M* Note: Stata dropped three strata with no sample units in the subpopulation.N 2NH 0h ? f 0`$( r S r SW H 0h ? f$ 0$( r STvP r S,w H 0h ? f80___PPT10.Fr$ 0$( r S@ r S @ H 0h ? f80___PPT10.F`3 0A9'2,( ,r , Sd 2,#"*@ ,, <a ?( O286 @` *, < ?@( Q1.590 @` (, < ?x(@ P4.82 @` &, < ?(x [ Unconditional @` , <@ ?"`q ( K106 @` , <, ?"`@q ( M1.576 @` , <@ ?"`xq @( L4.82 @` , <@ ?q x( PScaled @` , <l> ?"`q K106 @` , < ?"`@q M1.576 @` , < ?"`x@q L4.82 @` , <, ?xq S Certainty @` , < ?"` K106 @` , <` ?"`@ M1.576 @` , <t ?"`x@ L4.82 @` , < ?x RCentered @` , < ?H K106 @` , <\ ?@H M1.576 @` , <l5 ?xH@ L4.82 @` , <> ?Hx T Missing SE @` , <H ?H W Design DF @` , <P ?@H TTSL SE @` , <Y ?x@H TEst. % @` , <b ?xH TMethod @`fB , 6o?`B , 01?HH`B , 01?`B , 01?`B , 01?q q fB , 6o? fB , 6o? `B !, 01?xx `B ", 01?@@ `B #, 01? fB $, 6o? ZB ', s*1 ?((H ,0h ? f80___PPT10.P$ 0$( r SD[ r S`R H 0h ? f80___PPT10.FHV- 0$( r Sz r Sh~Pp H 0h ? f$ 0$( r SP r S< H 0h ? f80___PPT10.GH@0$ 0$( r S r S H 0h ? f80___PPT10.GH߮ 0 $( r SnP r SH_ H 0h ? fr0`P_ RaTc ,g)@ei4} 8$60BZ:(<pl@E<h.#Y5s X M1RP( L/0LDTimes New RomanPPs:ؖO0ؖ@Oh+'0`h ,8@TStatistical and Methodological Issues in the Analysis of Complex Sample Survey DataSak Westbwest146Microsoft PowerPoint@Q@0pյ@pba]Ghg ,V'&" WMFC! <8 ?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ !"#$%&'()*+,-./012345789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~Root EntrydO)`cPictures4ICurrent User/SummaryInformation(6PowerPoint Document([DocumentSummaryInformation8. @n?" dd@ @@``|DR1% ' ! ("#$%&'(3)*.?$$b$cjs^bd;4I0AA@uʚ;2Nʚ;g4JdJdD:O0zppp@<4!d!dv0Pls:<4ddddv0Pls:0___PPT10 ___PPT9d\E?~- j2009 Stata Conference: Subpop Analysis of Survey DataO =HEUsing Stata for Subpopulation Analysis of Complex Sample Survey DatajBrady T. West PhD Student Michigan Program in Survey Methodology July 30, 2009 2009 Stata Conference(kA)Presentation OutlineIntroduction: Subclass Analysis Issues Kish s Taxonomy of Subclasses Two Alternative Approaches to Inference Variance Estimation and Methods for Singletons Examples using NHANES and NHAMCS Data Suggestions for Practice Directions for Future Research" Subclass Analysis IssuesAnalysts of large, complex sample survey data sets are often interested in making inferences about subpopulations of the original population that the sample was selected from (e.g., Caucasian Females) These subpopulations are referred to interchangeably in various literatures as subgroups, subclasses, subpopulations, domains, and subdomains, leading to confusion among analysts of survey datajZB ]L 5)&@Subclass Analysis Issues, cont dJSoftware procedures for analysis of complex sample survey data are becoming more powerful, flexible, and widely available, offering analysts several options Analysts need to be careful when analyzing subclasses, and be aware of the alternative approaches to subclass analysis that are possible and their implications for inference :Kish s Taxonomy of SubclassesDesign Domains: Restricted to specific strata according to the complex sample design (usually geographically, e.g., Texas) Cross-Classes: Broadly distributed (in theory) across the strata and primary sampling units defining a complex sample (e.g., African-Americans over age 50) Mixed Classes: Disproportionately distributed across the complex sample design (e.g., Hispanics in a sample including Los Angeles as a stratum) See Kish (1987), Statistical Design for ResearchZN r *(-Design DomainsX = Sample Element in Subclass..+) Cross-Classes-* Mixed Classes,'0Applying Kish s TaxonomyThe type of subclass is critical for determining an appropriate analysis approach Two possible approaches to inference motivated by the taxonomy: 1. Unconditional approach (cross-classes, mixed classes) 2. Conditional approach (design domains) pZeZZv-.+The Unconditional ApproachQAppropriate for Cross-Classes, and in some cases Mixed Classes; the subclass of interest theoretically can appear in all design strata and primary sampling units (PSUs) KEY POINT: Allow the software to process the entire survey data set, and recognize all possible design strata and PSUs; DO NOT delete sample cases not in the subclass! R k #D*,t3/,The Unconditional ApproachRationale: estimated variances for sample estimates of subclass parameters (based on within-stratum variance between PSUs) need to reflect sample-to-sample variability based on the full complex design In other words, if a particular subclass does not appear in a PSU in any given sample (although in theory it could have), that PSU should contribute 0 to variance estimates, rather than be ignored completely!VZ !u!=:The Unconditional ApproachFurther, the subclass sample size in each stratum is going to be a random variable, and theoretical sample-to-sample variance in realizations of this random variable should be incorporated into any variance estimation procedures4C0-The Unconditional ApproachIf cross-classes (or in some cases mixed classes) are being analyzed, and PSUs where the subclass does not appear (by random chance) are deleted, problems arise Some strata may appear to have only one PSU by design (preventing variance estimation unless an ad hoc approach is used) Entire design strata may be dropped, impacting variance estimates and calculations of degrees of freedomrZ YJ6EB/The Unconditional Approach: General Stata Code00(Asvy, subpop(indicator): command varlist, options indicator = an indicator variable for the subpop or an if condition, e.g., if male == 1 svy: mean, over(groupvar) svy: prop, over(groupvar) Stata drops strata* with no subpopulation observations from degrees of freedom calculations * Exercise: repeat 10 times really fastf) (4( 1.The Conditional ApproachAppropriate for Design Domains, where a subclass cannot appear outside of specific design strata The rationale behind the unconditional approach no longer applies Certain design strata should not contribute to variance estimation or calculation of degrees of freedom @ZI2/The Conditional ApproachKRestrict the analysis to only those design strata where the subclass of interest exists Variance estimates reflecting sample-to-sample variability should only be based on those design strata where the subclass can appear (unlike the unconditional approach) Subclass sample sizes in design domains are assumed to be fixed, by designLLZVFC-The Conditional Approach: General Stata Code..(svy: command varlist if (condition), options (condition) might be male == 1, or a more complex combination of conditions (e.g., male == 1 & age >= 50 & age <= 90)T j$ GDVariance Estimation Methods%All of these issues are only relevant when using Taylor Series Linearization, which is a default for variance estimation in Stata Conditional analyses are OK to perform when using replication methods, such as Balanced Repeated Replication or Jackknife Repeated Replication (Rust and Rao, 1996)HEfAd-hoc Fixes for Singleton Clusters in Stata 10.144(JStata 10.1 provides users with four ad-hoc fixes for the problem where strata are identified with only a single ultimate cluster for variance estimation in a subpopulation analysis: Report Missing Standard Errors (not really a fix) Treat Units as Certainty Units, which contribute nothing to the standard error Scale Variance using Certainty Units, which uses the average variance from each stratum with multiple PSUs for each stratum with only a single PSU Center at the Grand Mean, where the variance contribution comes from a deviation from the grand mean instead of the stratum meanP" P1$oi30Example: The NHANES Data+We first consider examples based on the NHANES II data set, collected from a nationally representative multistage probability sample of the U.S. population from 1976-1980 Briefly, a sample of the U.S. population was given medical examinations in an effort to assess the health of the U.S. population41Example NHANES Analysis<Analysis Subclass: African-Americans ages 50 and above (this is a cross-class of the U.S. population, which can theoretically appear in all design strata and PSUs) Analysis Objective: Estimate the mean systolic blood pressure of this subclass and an appropriate standard error See West et al. (2007) for more detailsn=Z1W=7IF4Conditional Approach:Stata Code for NHANES Analysis55(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(centered) Also singleunit(certainty), singleunit(scaled) gen b50subp = (race == 2 & ager >= 50) svy: mean bpsyst if b50subp == 1 PZ22g )MIConditional Approach: Results63Conditional Approach?This approach would not be appropriate for this particular subclass Computed standard errors would generally be biased downward, because additional sources of sample-to-sample variability are ignored when following this approach Same issues apply for analytic models Evidence that the scaled ad-hoc fix may be overly conservative!HMZwADBJG6Unconditional Approach:Stata Code for NHANES Analysis77(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) Note: choice of single unit option does not matter when following this approach! gen b50subp = (race == 2 & ager >= 50) svy, subpop(b50subp): mean bpsyst&2eIz74Unconditional Approach: Results85Unconditional Approach?8This approach would be the appropriate choice for a cross-class such as African-Americans over the age of 50 Inferences are theoretically appropriate Same idea for analytic models Results suggest that the centered and certainty ad-hoc fixes for conditional analyses are reasonable>;Example: The NHAMCS DataAnalysis Subclass: Visits to Emergency Departments (ED) by African-American men ages 60 and above (this is another cross-class of the U.S. population, which can theoretically appear in all NHAMCS design strata and PSUs) Analysis Objective: Estimate the percentage of all ED visits by members of this subclass for dizziness and/or vertigo in 2004 See West et al. (2008) for more details b^R(?<Stata Code for NHAMCS Analyses(svyset cpsum [pweight = patwt], strata(cstratm) singleunit(& ) generate subc = (settype == 3 & sex == 2 & agecat == 5 & race == 2) svy: tabulate dizzyrfv if subc == 1, se ci percent * conditional svy, subpop(subc): tabulate dizzyrfv, se ci percent * unconditional 4Z42 KHNHAMCS Analysis Results@=NHAMCS Analysis Implications!No problems with strata having only a single ultimate cluster: ad-hoc fixes all give the same results Weighted point estimates are identical Substantially fewer design-based degrees of freedom when following the conditional approach; the full complex design will not be reflected in estimation of sample-to-sample variance (many ultimate clusters are lost) Conditional analysis assumes that each sample will be of fixed size n = 397 for variance estimation purposes; no random variance! Conditional analysis results in overly liberal inferences d"Zd:7Suggestions for PracticeConsider Kish s Taxonomy when determining an appropriate subclass analysis approach Utilize the appropriate software options for unconditional analyses when analyzing cross-classes Be careful with missing values when creating the subpopulation indicator The unconditional analysis approach generally works fine for both cases (when in doubt, use this approach)iPiA>Directions for Future ResearchMore appropriate calculation / estimation of design-based and effective degrees of freedom for sparse subclasses Development of analytic theory for interval estimation when working with small subclasses, which does not rely on asymptotic results B? ReferencesNKish, L. 1987. Statistical Design for Research. New York: Wiley. Rust, K. F., and J. N. K. Rao. 1996. Variance estimation for complex surveys using replication. Statistical Methods in Medical Research 5: 283 310. West, B.T., Berglund, P., and Heeringa, S.G. 2008. A Closer Examination of Subpopulation Analysis of Complex Sample Survey Data. The Stata Journal, 8(3), 1-12. West, B.T., Berglund, P., and Heeringa, S.G. 2007. Alternative Approaches to Subclass Analysis of Complex Sample Survey Data. Proceedings of the 2007 Joint Statistical Meetings. ,(Z(_>[<9Questions / Thank You!For additional questions, comments, or electronic copies of these slides or the papers, please send an email to bwest@umich.edup 0$( r Sz r Sh~Pp H 0h ? fr= 50 & age <= 90)T j$ GDVariance Estimation Methods%All of these issues are only relevant when using Taylor Series Linearization, which is a default for variance estimation in Stata Conditional analyses are OK to perform when using replication methods, such as Balanced Repeated Replication or Jackknife Repeated Replication (Rust and Rao, 1996)HEfAd-hoc Fixes for Singleton Clusters in Stata 10.144(JStata 10.1 provides users with four ad-hoc fixes for the problem where strata are identified with only a single ultimate cluster for variance estimation in a subpopulation analysis: Report Missing Standard Errors (not really a fix) Treat Units as Certainty Units, which contribute nothing to the standard error Scale Variance using Certainty Units, which uses the average variance from each stratum with multiple PSUs for each stratum with only a single PSU Center at the Grand Mean, where the variance contribution comes from a deviation from the grand mean instead of the stratum meanP" P1$oi30Example: The NHANES Data+We first consider examples based on the NHANES II data set, collected from a nationally representative multistage probability sample of the U.S. population from 1976-1980 Briefly, a sample of the U.S. population was given medical examinations in an effort to assess the health of the U.S. population41Example NHANES Analysis<Analysis Subclass: African-Americans ages 50 and above (this is a cross-class of the U.S. population, which can theoretically appear in all design strata and PSUs) Analysis Objective: Estimate the mean systolic blood pressure of this subclass and an appropriate standard error See West et al. (2007) for more detailsn=Z1W=7IF4Conditional Approach:Stata Code for NHANES Analysis55(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(centered) Also singleunit(certainty), singleunit(scaled) gen b50subp = (race == 2 & ager >= 50) svy: mean bpsyst if b50subp == 1 PZ22g )MIConditional Approach: Results63Conditional Approach?This approach would not be appropriate for this particular subclass Computed standard errors would generally be biased downward, because additional sources of sample-to-sample variability are ignored when following this approach Same issues apply for analytic models Evidence that the scaled ad-hoc fix may be overly conservative!HMZwADBJG6Unconditional Approach:Stata Code for NHANES Analysis77(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) Note: choice of single unit option does not matter when following this approach! gen b50subp = (race == 2 & ager >= 50) svy, subpop(b50subp): mean bpsyst&2eIz74Unconditional Approach: Results85Unconditional Approach?8This approach would be the appropriate choice for a cross-class such as African-Americans over the age of 50 Inferences are theoretically appropriate Same idea for analytic models Results suggest that the centered and certainty ad-hoc fixes for conditional analyses are reasonable>;Example: The NHAMCS DataAnalysis Subclass: Visits to Emergency Departments (ED) by African-American men ages 60 and above (this is another cross-class of the U.S. population, which can theoretically appear in all NHAMCS design strata and PSUs) Analysis Objective: Estimate the percentage of all ED visits by members of this subclass for dizziness and/or vertigo in 2004 See West et al. (2008) for more details b^R(?<Stata Code for NHAMCS Analyses(svyset cpsum [pweight = patwt], strata(cstratm) singleunit(& ) generate subc = (settype == 3 & sex == 2 & agecat == 5 & race == 2) svy: tabulate dizzyrfv if subc == 1, se ci percent * conditional svy, subpop(subc): tabulate dizzyrfv, se ci percent * unconditional 4Z42 KHNHAMCS Analysis Results@=NHAMCS Analysis Implications!No problems with strata having only a single ultimate cluster: ad-hoc fixes all give the same results Weighted point estimates are identical Substantially fewer design-based degrees of freedom when following the conditional approach; the full complex design will not be reflected in estimation of sample-to-sample variance (many ultimate clusters are lost) Conditional analysis assumes that each sample will be of fixed size n = 397 for variance estimation purposes; no random variance! Conditional analysis results in overly liberal inferences d"Zd:7Suggestions for PracticeConsider Kish s Taxonomy when determining an appropriate subclass analysis approach Utilize the appropriate software options for unconditional analyses when analyzing cross-classes Be careful with missing values when creating the subpopulation indicator The unconditional analysis approach generally works fine for both cases (when in doubt, use this approach)iPiA>Directions for Future ResearchMore appropriate calculation / estimation of design-based and effective degrees of freedom for sparse subclasses or mixed classes Development of analytic theory for interval estimation when working with small subclasses, which does not rely on asymptotic results B? ReferencesNKish, L. 1987. Statistical Design for Research. New York: Wiley. Rust, K. F., and J. N. K. Rao. 1996. Variance estimation for complex surveys using replication. Statistical Methods in Medical Research 5: 283 310. West, B.T., Berglund, P., and Heeringa, S.G. 2008. A Closer Examination of Subpopulation Analysis of Complex Sample Survey Data. The Stata Journal, 8(3), 1-12. West, B.T., Berglund, P., and Heeringa, S.G. 2007. Alternative Approaches to Subclass Analysis of Complex Sample Survey Data. Proceedings of the 2007 Joint Statistical Meetings. ,(Z(_>[<9Questions / Thank You!For additional questions, comments, or electronic copies of these slides or the papers, please send an email to bwest@umich.edup$ 0$( r SP r S< H 0h ? f80___PPT10.GH@0rsA)>s sUM1cP( L/0LDTimes New RomanPPs:ؖO0ؖ@. @n?" dd@ @@``|DR1% ' ! ("#$%&'(3)*.?$$b$cjs^bd;4I0AA@uʚ;2Nʚ;g4JdJdD:O0zppp@<4!d!dv0Pls:<4ddddv0Pls:0___PPT10 ___PPT9d\E?~- j2009 Stata Conference: Subpop Analysis of Survey DataO =HEUsing Stata for Subpopulation Analysis of Complex Sample Survey DatajBrady T. West PhD Student Michigan Program in Survey Methodology July 30, 2009 2009 Stata Conference(kA)Presentation OutlineIntroduction: Subclass Analysis Issues Kish s Taxonomy of Subclasses Two Alternative Approaches to Inference Variance Estimation and Methods for Singletons Examples using NHANES and NHAMCS Data Suggestions for Practice Directions for Future Research" Subclass Analysis IssuesAnalysts of large, complex sample survey data sets are often interested in making inferences about subpopulations of the original population that the sample was selected from (e.g., Caucasian Females) These subpopulations are referred to interchangeably in various literatures as subgroups, subclasses, subpopulations, domains, and subdomains, leading to confusion among analysts of survey datajZB ]L 5)&@Subclass Analysis Issues, cont dJSoftware procedures for analysis of complex sample survey data are becoming more powerful, flexible, and widely available, offering analysts several options Analysts need to be careful when analyzing subclasses, and be aware of the alternative approaches to subclass analysis that are possible and their implications for inference :Kish s Taxonomy of SubclassesDesign Domains: Restricted to specific strata according to the complex sample design (usually geographically, e.g., Texas) Cross-Classes: Broadly distributed (in theory) across the strata and primary sampling units defining a complex sample (e.g., African-Americans over age 50) Mixed Classes: Disproportionately distributed across the complex sample design (e.g., Hispanics in a sample including Los Angeles as a stratum) See Kish (1987), Statistical Design for ResearchZN r *(-Design DomainsX = Sample Element in Subclass..+) Cross-Classes-* Mixed Classes,'0Applying Kish s TaxonomyThe type of subclass is critical for determining an appropriate analysis approach Two possible approaches to inference motivated by the taxonomy: 1. Unconditional approach (cross-classes, mixed classes) 2. Conditional approach (design domains) pZeZZv-.+The Unconditional ApproachQAppropriate for Cross-Classes, and in some cases Mixed Classes; the subclass of interest theoretically can appear in all design strata and primary sampling units (PSUs) KEY POINT: Allow the software to process the entire survey data set, and recognize all possible design strata and PSUs; DO NOT delete sample cases not in the subclass! R k #D*,t3/,The Unconditional ApproachRationale: estimated variances for sample estimates of subclass parameters (based on within-stratum variance between PSUs) need to reflect sample-to-sample variability based on the full complex design In other words, if a particular subclass does not appear in a PSU in any given sample (although in theory it could have), that PSU should contribute 0 to variance estimates, rather than be ignored completely!VZ !u!=:The Unconditional ApproachFurther, the subclass sample size in each stratum is going to be a random variable, and theoretical sample-to-sample variance in realizations of this random variable should be incorporated into any variance estimation procedures4C0-The Unconditional ApproachIf cross-classes (or in some cases mixed classes) are being analyzed, and PSUs where the subclass does not appear (by random chance) are deleted, problems arise Some strata may appear to have only one PSU by design (preventing variance estimation unless an ad hoc approach is used) Entire design strata may be dropped, impacting variance estimates and calculations of degrees of freedomrZ YJ6EB/The Unconditional Approach: General Stata Code00(Asvy, subpop(indicator): command varlist, options indicator = an indicator variable for the subpop or an if condition, e.g., if male == 1 svy: mean, over(groupvar) svy: prop, over(groupvar) Stata drops strata* with no subpopulation observations from degrees of freedom calculations * Exercise: repeat 10 times really fastf) (4( !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}1.The Conditional ApproachAppropriate for Design Domains, where a subclass cannot appear outside of specific design strata The rationale behind the unconditional approach no longer applies Certain design strata should not contribute to variance estimation or calculation of degrees of freedom @ZI2/The Conditional ApproachKRestrict the analysis to only those design strata where the subclass of interest exists Variance estimates reflecting sample-to-sample variability should only be based on those design strata where the subclass can appear (unlike the unconditional approach) Subclass sample sizes in design domains are assumed to be fixed, by designLLZVFC-The Conditional Approach: General Stata Code..(svy: command varlist if (condition), options (condition) might be male == 1, or a more complex combination of conditions (e.g., male == 1 & age >= 50 & age <= 90)T j$ GDVariance Estimation Methods%All of these issues are only relevant when using Taylor Series Linearization, which is a default for variance estimation in Stata Conditional analyses are OK to perform when using replication methods, such as Balanced Repeated Replication or Jackknife Repeated Replication (Rust and Rao, 1996)HEfAd-hoc Fixes for Singleton Clusters in Stata 10.144(JStata 10.1 provides users with four ad-hoc fixes for the problem where strata are identified with only a single ultimate cluster for variance estimation in a subpopulation analysis: Report Missing Standard Errors (not really a fix) Treat Units as Certainty Units, which contribute nothing to the standard error Scale Variance using Certainty Units, which uses the average variance from each stratum with multiple PSUs for each stratum with only a single PSU Center at the Grand Mean, where the variance contribution comes from a deviation from the grand mean instead of the stratum meanP" P1$oi30Example: The NHANES Data+We first consider examples based on the NHANES II data set, collected from a nationally representative multistage probability sample of the U.S. population from 1976-1980 Briefly, a sample of the U.S. population was given medical examinations in an effort to assess the health of the U.S. population41Example NHANES Analysis<Analysis Subclass: African-Americans ages 50 and above (this is a cross-class of the U.S. population, which can theoretically appear in all design strata and PSUs) Analysis Objective: Estimate the mean systolic blood pressure of this subclass and an appropriate standard error See West et al. (2007) for more detailsn=Z1W=7IF4Conditional Approach:Stata Code for NHANES Analysis55(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(centered) Also singleunit(certainty), singleunit(scaled) gen b50subp = (race == 2 & ager >= 50) svy: mean bpsyst if b50subp == 1 PZ22g )MIConditional Approach: Results63Conditional Approach?This approach would not be appropriate for this particular subclass Computed standard errors would generally be biased downward, because additional sources of sample-to-sample variability are ignored when following this approach Same issues apply for analytic models Evidence that the scaled ad-hoc fix may be overly conservative!HMZwADBJG6Unconditional Approach:Stata Code for NHANES Analysis77(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) Note: choice of single unit option does not matter when following this approach! gen b50subp = (race == 2 & ager >= 50) svy, subpop(b50subp): mean bpsyst&2eIz74Unconditional Approach: Results85Unconditional Approach?8This approach would be the appropriate choice for a cross-class such as African-Americans over the age of 50 Inferences are theoretically appropriate Same idea for analytic models Results suggest that the centered and certainty ad-hoc fixes for conditional analyses are reasonable>;Example: The NHAMCS DataAnalysis Subclass: Visits to Emergency Departments (ED) by African-American men ages 60 and above (this is another cross-class of the U.S. population, which can theoretically appear in all NHAMCS design strata and PSUs) Analysis Objective: Estimate the percentage of all ED visits by members of this subclass for dizziness and/or vertigo in 2004 See West et al. (2008) for more details b^R(?<Stata Code for NHAMCS Analyses(svyset cpsum [pweight = patwt], strata(cstratm) singleunit(& ) generate subc = (settype == 3 & sex == 2 & agecat == 5 & race == 2) svy: tabulate dizzyrfv if subc == 1, se ci percent * conditional svy, subpop(subc): tabulate dizzyrfv, se ci percent * unconditional 4Z42 KHNHAMCS Analysis Results@=NHAMCS Analysis Implications!No problems with strata having only a single ultimate cluster: ad-hoc fixes all give the same results Weighted point estimates are identical Substantially fewer design-based degrees of freedom when following the conditional approach; the full complex design will not be reflected in estimation of sample-to-sample variance (many ultimate clusters are lost) Conditional analysis assumes that each sample will be of fixed size n = 397 for variance estimation purposes; no random variance! Conditional analysis results in overly liberal inferences d"Zd:7Suggestions for PracticeConsider Kish s Taxonomy when determining an appropriate subclass analysis approach Utilize the appropriate software options for unconditional analyses when analyzing cross-classes Be careful with missing values when creating the subpopulation indicator The unconditional analysis approach generally works fine for both cases (when in doubt, use this approach)iPiA>Directions for Future ResearchMore appropriate calculation / estimation of design-based and effective degrees of freedom for sparse subclasses or mixed classes Development of analytic theory for interval estimation when working with small subclasses, which does not rely on asymptotic results B? ReferencesNKish, L. 1987. Statistical Design for Research. New York: Wiley. Rust, K. F., and J. N. K. Rao. 1996. Variance estimation for complex surveys using replication. Statistical Methods in Medical Research 5: 283 310. West, B.T., Berglund, P., and Heeringa, S.G. 2008. A Closer Examination of Subpopulation Analysis of Complex Sample Survey Data. The Stata Journal, 8(3), 1-12. West, B.T., Berglund, P., and Heeringa, S.G. 2007. Alternative Approaches to Subclass Analysis of Complex Sample Survey Data. Proceedings of the 2007 Joint Statistical Meetings. ,(Z(_>[<9Questions / Thank You!For additional questions, comments, or electronic copies of these slides or the papers, please send an email to bwest@umich.edupr?s mM1xP( L/0LDTimes New RomanPPs:ؖO0ؖ@. @n?" dd@ @@``|DR1% ' ! ("#$%&'(3)*.?$$b$cjs^bd;4I0AA@uʚ;2Nʚ;g4JdJdD:O0zppp@<4!d!dv0Pls:<4ddddv0Pls:0___PPT10 ___PPT9d\E?~- j2009 Stata Conference: Subpop Analysis of Survey DataO =HEUsing Stata for Subpopulation Analysis of Complex Sample Survey DatajBrady T. West PhD Student Michigan Program in Survey Methodology July 30, 2009 2009 Stata Conference(kA)Presentation OutlineIntroduction: Subclass Analysis Issues Kish s Taxonomy of Subclasses Two Alternative Approaches to Inference Variance Estimation and Methods for Singletons Examples using NHANES and NHAMCS Data Suggestions for Practice Directions for Future Research" Subclass Analysis IssuesAnalysts of large, complex sample survey data sets are often interested in making inferences about subpopulations of the original population that the sample was selected from (e.g., Caucasian Females) These subpopulations are referred to interchangeably in various literatures as subgroups, subclasses, subpopulations, domains, and subdomains, leading to confusion among analysts of survey datajZB ]L 5)&@Subclass Analysis Issues, cont dJSoftware procedures for analysis of complex sample survey data are becoming more powerful, flexible, and widely available, offering analysts several options Analysts need to be careful when analyzing subclasses, and be aware of the alternative approaches to subclass analysis that are possible and their implications for inference :Kish s Taxonomy of SubclassesDesign Domains: Restricted to specific strata according to the complex sample design (usually geographically, e.g., Texas) Cross-Classes: Broadly distributed (in theory) across the strata and primary sampling units defining a complex sample (e.g., African-Americans over age 50) Mixed Classes: Disproportionately distributed across the complex sample design (e.g., Hispanics in a sample including Los Angeles as a stratum) See Kish (1987), Statistical Design for ResearchZN r *(-Design DomainsX = Sample Element in Subclass..+) Cross-Classes-* Mixed Classes,'0Applying Kish s TaxonomyThe type of subclass is critical for determining an appropriate analysis approach Two possible approaches to inference motivated by the taxonomy: 1. Unconditional approach (cross-classes, mixed classes) 2. Conditional approach (design domains) pZeZZv-.+The Unconditional ApproachQAppropriate for Cross-Classes, and in some cases Mixed Classes; the subclass of interest theoretically can appear in all design strata and primary sampling units (PSUs) KEY POINT: Allow the software to process the entire survey data set, and recognize all possible design strata and PSUs; DO NOT delete sample cases not in the subclass! R k #D*,t3/,The Unconditional ApproachRationale: estimated variances for sample estimates of subclass parameters (based on within-stratum variance between PSUs) need to reflect sample-to-sample variability based on the full complex design In other words, if a particular subclass does not appear in a PSU in any given sample (although in theory it could have), that PSU should contribute 0 to variance estimates, rather than be ignored completely!VZ !u!=:The Unconditional ApproachFurther, the subclass sample size in each stratum is going to be a random variable, and theoretical sample-to-sample variance in realizations of this random variable should be incorporated into any variance estimation procedures4C0-The Unconditional ApproachIf cross-classes (or in some cases mixed classes) are being analyzed, and PSUs where the subclass does not appear (by random chance) are deleted, problems arise Some strata may appear to have only one PSU by design (preventing variance estimation unless an ad hoc approach is used) Entire design strata may be dropped, impacting variance estimates and calculations of degrees of freedomrZ YJ6EB/The Unconditional Approach: General Stata Code00(Asvy, subpop(indicator): command varlist, options indicator = an indicator variable for the subpop or an if condition, e.g., if male == 1 svy: mean, over(groupvar) svy: prop, over(groupvar) Stata drops strata* with no subpopulation observations from degrees of freedom calculations * Exercise: repeat 10 times really fastf) (4( 1.The Conditional ApproachAppropriate for Design Domains, where a subclass cannot appear outside of specific design strata The rationale behind the unconditional approach no longer applies Certain design strata should not contribute to variance estimation or calculation of degrees of freedom @ZI2/The Conditional ApproachKRestrict the analysis to only those design strata where the subclass of interest exists Variance estimates reflecting sample-to-sample variability should only be based on those design strata where the subclass can appear (unlike the unconditional approach) Subclass sample sizes in design domains are assumed to be fixed, by designLLZVFC-The Conditional Approach: General Stata Code..(svy: command varlist if (condition), options (condition) might be male == 1, or a more complex combination of conditions (e.g., male == 1 & age >= 50 & age <= 90)T j$ GDVariance Estimation Methods%All of these issues are only relevant when using Taylor Series Linearization, which is a default for variance estimation in Stata Conditional analyses are OK to perform when using replication methods, such as Balanced Repeated Replication or Jackknife Repeated Replication (Rust and Rao, 1996)HEfAd-hoc Fixes for Singleton Clusters in Stata 10.144(JStata 10.1 provides users with four ad-hoc fixes for the problem where strata are identified with only a single ultimate cluster for variance estimation in a subpopulation analysis: Report Missing Standard Errors (not really a fix) Treat Units as Certainty Units, which contribute nothing to the standard error Scale Variance using Certainty Units, which uses the average variance from each stratum with multiple PSUs for each stratum with only a single PSU Center at the Grand Mean, where the variance contribution comes from a deviation from the grand mean instead of the stratum meanP" P1$oi30Example: The NHANES Data@We first consider examples based on the NHANES II data set, collected from a nationally representative multistage probability sample of the U.S. population from 1976-1980 (oldie but a goodie) Briefly, a sample of the U.S. population was given medical examinations in an effort to assess the health of the U.S. population41Example NHANES Analysis<Analysis Subclass: African-Americans ages 50 and above (this is a cross-class of the U.S. population, which can theoretically appear in all design strata and PSUs) Analysis Objective: Estimate the mean systolic blood pressure of this subclass and an appropriate standard error See West et al. (2007) for more detailsn=Z1W=7IF4Conditional Approach:Stata Code for NHANES Analysis55(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(centered) Also singleunit(certainty), singleunit(scaled) gen b50subp = (race == 2 & ager >= 50) svy: mean bpsyst if b50subp == 1 PZ22g )MIConditional Approach: Results63Conditional Approach?This approach would not be appropriate for this particular subclass Computed standard errors would generally be biased downward, because additional sources of sample-to-sample variability are ignored when following this approach Same issues apply for analytic models Evidence that the scaled ad-hoc fix may be overly conservative!HMZwADBJG6Unconditional Approach:Stata Code for NHANES Analysis77(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) Note: choice of single unit option does not matter when following this approach! gen b50subp = (race == 2 & ager >= 50) svy, subpop(b50subp): mean bpsyst&2eIz74Unconditional Approach: Results85Unconditional Approach?8This approach would be the appropriate choice for a cross-class such as African-Americans over the age of 50 Inferences are theoretically appropriate Same idea for analytic models Results suggest that the centered and certainty ad-hoc fixes for conditional analyses are reasonable>;Example: The NHAMCS DataAnalysis Subclass: Visits to Emergency Departments (ED) by African-American men ages 60 and above (this is another cross-class of the U.S. population, which can theoretically appear in all NHAMCS design strata and PSUs) Analysis Objective: Estimate the percentage of all ED visits by members of this subclass for dizziness and/or vertigo in 2004 See West et al. (2008) for more details b^R(?<Stata Code for NHAMCS Analyses(svyset cpsum [pweight = patwt], strata(cstratm) singleunit(& ) generate subc = (settype == 3 & sex == 2 & agecat == 5 & race == 2) svy: tabulate dizzyrfv if subc == 1, se ci percent * conditional svy, subpop(subc): tabulate dizzyrfv, se ci percent * unconditional 4Z42 KHNHAMCS Analysis Results@=NHAMCS Analysis Implications!No problems with strata having only a single ultimate cluster: ad-hoc fixes all give the same results Weighted point estimates are identical Substantially fewer design-based degrees of freedom when following the conditional approach; the full complex design will not be reflected in estimation of sample-to-sample variance (many ultimate clusters are lost) Conditional analysis assumes that each sample will be of fixed size n = 397 for variance estimation purposes; no random variance! Conditional analysis results in overly liberal inferences d"Zd:7Suggestions for PracticeConsider Kish s Taxonomy when determining an appropriate subclass analysis approach Utilize the appropriate software options for unconditional analyses when analyzing cross-classes Be careful with missing values when creating the subpopulation indicator The unconditional analysis approach generally works fine for both cases (when in doubt, use this approach)iPiA>Directions for Future ResearchMore appropriate calculation / estimation of design-based and effective degrees of freedom for sparse subclasses or mixed classes Development of analytic theory for interval estimation when working with small subclasses, which does not rely on asymptotic results B? ReferencesNKish, L. 1987. Statistical Design for Research. New York: Wiley. Rust, K. F., and J. N. K. Rao. 1996. Variance estimation for complex surveys using replication. Statistical Methods in Medical Research 5: 283 310. West, B.T., Berglund, P., and Heeringa, S.G. 2008. A Closer Examination of Subpopulation Analysis of Complex Sample Survey Data. The Stata Journal, 8(3), 1-12. West, B.T., Berglund, P., and Heeringa, S.G. 2007. Alternative Approaches to Subclass Analysis of Complex Sample Survey Data. Proceedings of the 2007 Joint Statistical Meetings. ,(Z(_>[<9Questions / Thank You!For additional questions, comments, or electronic copies of these slides or the papers, please send an email to bwest@umich.edup 0$( r S P r S Pp H 0h ? fr03g0s iM1xP( L/0LDTimes New RomanPPs:ؖO0ؖ@. @n?" dd@ @@``|DR1% ' ! ("#$%&'(3)*.?$$b$cjs^bd;4I0AA@uʚ;2Nʚ;g4JdJdD:O0zppp@<4!d!dv0Pls:<4ddddv0Pls:0___PPT10 ___PPT9d\E?~- j2009 Stata Conference: Subpop Analysis of Survey DataO =HEUsing Stata for Subpopulation Analysis of Complex Sample Survey DatajBrady T. West PhD Student Michigan Program in Survey Methodology July 30, 2009 2009 Stata Conference(kA)Presentation OutlineIntroduction: Subclass Analysis Issues Kish s Taxonomy of Subclasses Two Alternative Approaches to Inference Variance Estimation and Methods for Singletons Examples using NHANES and NHAMCS Data Suggestions for Practice Directions for Future Research" Subclass Analysis IssuesAnalysts of large, complex sample survey data sets are often interested in making inferences about subpopulations of the original population that the sample was selected from (e.g., Caucasian Females) These subpopulations are referred to interchangeably in various literatures as subgroups, subclasses, subpopulations, domains, and subdomains, leading to confusion among analysts of survey datajZB ]L 5)&@Subclass Analysis Issues, cont dJSoftware procedures for analysis of complex sample survey data are becoming more powerful, flexible, and widely available, offering analysts several options Analysts need to be careful when analyzing subclasses, and be aware of the alternative approaches to subclass analysis that are possible and their implications for inference :Kish s Taxonomy of SubclassesDesign Domains: Restricted to specific strata according to the complex sample design (usually geographically, e.g., Texas) Cross-Classes: Broadly distributed (in theory) across the strata and primary sampling units defining a complex sample (e.g., African-Americans over age 50) Mixed Classes: Disproportionately distributed across the complex sample design (e.g., Hispanics in a sample including Los Angeles as a stratum) See Kish (1987), Statistical Design for ResearchZN r *(-Design DomainsX = Sample Element in Subclass..+) Cross-Classes-* Mixed Classes,'0Applying Kish s TaxonomyThe type of subclass is critical for determining an appropriate analysis approach Two possible approaches to inference motivated by the taxonomy: 1. Unconditional approach (cross-classes, mixed classes) 2. Conditional approach (design domains) pZeZZv-.+The Unconditional ApproachQAppropriate for Cross-Classes, and in some cases Mixed Classes; the subclass of interest theoretically can appear in all design strata and primary sampling units (PSUs) KEY POINT: Allow the software to process the entire survey data set, and recognize all possible design strata and PSUs; DO NOT delete sample cases not in the subclass! R k #D*,t3/,The Unconditional ApproachRationale: estimated variances for sample estimates of subclass parameters (based on within-stratum variance between PSUs) need to reflect sample-to-sample variability based on the full complex design In other words, if a particular subclass does not appear in a PSU in any given sample (although in theory it could have), that PSU should contribute 0 to variance estimates, rather than be ignored completely!VZ !u!=:The Unconditional ApproachFurther, the subclass sample size in each stratum is going to be a random variable, and theoretical sample-to-sample variance in realizations of this random variable should be incorporated into any variance estimation procedures4C0-The Unconditional ApproachIf cross-classes (or in some cases mixed classes) are being analyzed, and PSUs where the subclass does not appear (by random chance) are deleted, problems arise Some strata may appear to have only one PSU by design (preventing variance estimation unless an ad hoc approach is used) Entire design strata may be dropped, impacting variance estimates and calculations of degrees of freedomrZ YJ6EB/The Unconditional Approach: General Stata Code00(Asvy, subpop(indicator): command varlist, options indicator = an indicator variable for the subpop or an if condition, e.g., if male == 1 svy: mean, over(groupvar) svy: prop, over(groupvar) Stata drops strata* with no subpopulation observations from degrees of freedom calculations * Exercise: repeat 10 times really fastf) (4( 1.The Conditional ApproachAppropriate for Design Domains, where a subclass cannot appear outside of specific design strata The rationale behind the unconditional approach no longer applies Certain design strata should not contribute to variance estimation or calculation of degrees of freedom @ZI2/The Conditional ApproachKRestrict the analysis to only those design strata where the subclass of interest exists Variance estimates reflecting sample-to-sample variability should only be based on those design strata where the subclass can appear (unlike the unconditional approach) Subclass sample sizes in design domains are assumed to be fixed, by designLLZVFC-The Conditional Approach: General Stata Code..(svy: command varlist if (condition), options (condition) might be male == 1, or a more complex combination of conditions (e.g., male == 1 & age >= 50 & age <= 90)T j$ GDVariance Estimation Methods%All of these issues are only relevant when using Taylor Series Linearization, which is a default for variance estimation in Stata Conditional analyses are OK to perform when using replication methods, such as Balanced Repeated Replication or Jackknife Repeated Replication (Rust and Rao, 1996)HEfAd-hoc Fixes for Singleton Clusters in Stata 10.144(JStata 10.1 provides users with four ad-hoc fixes for the problem where strata are identified with only a single ultimate cluster for variance estimation in a subpopulation analysis: Report Missing Standard Errors (not really a fix) Treat Units as Certainty Units, which contribute nothing to the standard error Scale Variance using Certainty Units, which uses the average variance from each stratum with multiple PSUs for each stratum with only a single PSU Center at the Grand Mean, where the variance contribution comes from a deviation from the grand mean instead of the stratum meanP" P1$oi30Example: The NHANES Data@We first consider examples based on the NHANES II data set, collected from a nationally representative multistage probability sample of the U.S. population from 1976-1980 (oldie but a goodie) Briefly, a sample of the U.S. population was given medical examinations in an effort to assess the health of the U.S. population41Example NHANES Analysis<Analysis Subclass: African-Americans ages 50 and above (this is a cross-class of the U.S. population, which can theoretically appear in all design strata and PSUs) Analysis Objective: Estimate the mean systolic blood pressure of this subclass and an appropriate standard error See West et al. (2007) for more detailsn=Z1W=7IF4Conditional Approach:Stata Code for NHANES Analysis55(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(centered) Also singleunit(certainty), singleunit(scaled) gen b50subp = (race == 2 & ager >= 50) svy: mean bpsyst if b50subp == 1 PZ22g )MIConditional Approach: Results63Conditional Approach?This approach would not be appropriate for this particular subclass Computed standard errors would generally be biased downward, because additional sources of sample-to-sample variability are ignored when following this approach Same issues apply for analytic models Evidence that the scaled ad-hoc fix may be overly conservative!HMZwADBJG6Unconditional Approach:Stata Code for NHANES Analysis77(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) Note: choice of single unit option does not matter when following this approach! gen b50subp = (race == 2 & ager >= 50) svy, subpop(b50subp): mean bpsyst&2eIz74Unconditional Approach: Results85Unconditional Approach?8This approach would be the appropriate choice for a cross-class such as African-Americans over the age of 50 Inferences are theoretically appropriate Same idea for analytic models Results suggest that the centered and certainty ad-hoc fixes for conditional analyses are reasonable>;Example: The NHAMCS DataAnalysis Subclass: Visits to Emergency Departments (ED) by African-American men ages 60 and above (this is another cross-class of the U.S. population, which can theoretically appear in all NHAMCS design strata and PSUs) Analysis Objective: Estimate the percentage of all ED visits by members of this subclass for dizziness and/or vertigo in 2004 See West et al. (2008) for more details b^R(?<Stata Code for NHAMCS Analyses(svyset cpsum [pweight = patwt], strata(cstratm) singleunit(& ) generate subc = (settype == 3 & sex == 2 & agecat == 5 & race == 2) svy: tabulate dizzyrfv if subc == 1, se ci percent * conditional svy, subpop(subc): tabulate dizzyrfv, se ci percent * unconditional 4Z42 KHNHAMCS Analysis Results@=NHAMCS Analysis Implications!No problems with strata having only a single ultimate cluster: ad-hoc fixes all give the same results Weighted point estimates are identical Substantially fewer design-based degrees of freedom when following the conditional approach; the full complex design will not be reflected in estimation of sample-to-sample variance (many ultimate clusters are lost) Conditional analysis assumes that each sample will be of fixed size n = 397 for variance estimation purposes; no random variance! Conditional analysis results in overly liberal inferences d"Zd:7Suggestions for PracticeConsider Kish s Taxonomy when determining an appropriate subclass analysis approach Utilize the appropriate software options for unconditional analyses when analyzing cross-classes Be careful with missing values when creating the subpopulation indicator The unconditional analysis approach generally works fine for both cases (when in doubt, use this approach)iPiA>Directions for Future ResearchMore appropriate calculation / estimation of design-based and effective degrees of freedom for sparse subclasses or mixed classes Development of analytic theory for interval estimation when working with small subclasses, which does not rely on asymptotic results B? ReferencesNKish, L. 1987. Statistical Design for Research. New York: Wiley. Rust, K. F., and J. N. K. Rao. 1996. Variance estimation for complex surveys using replication. Statistical Methods in Medical Research 5: 283 310. West, B.T., Berglund, P., and Heeringa, S.G. 2008. A Closer Examination of Subpopulation Analysis of Complex Sample Survey Data. The Stata Journal, 8(3), 1-12. West, B.T., Berglund, P., and Heeringa, S.G. 2007. Alternative Approaches to Subclass Analysis of Complex Sample Survey Data. Proceedings of the 2007 Joint Statistical Meetings. ,(Z(_>[<9Questions / Thank You!For additional questions, comments, or electronic copies of these slides or the papers, please send an email to bwest@umich.edupris iXM1xP( L/0LDTimes New RomanPPs:ؖO0ؖ@. @n?" dd@ @@``|DR1% ' ! ("#$%&'(3)*.?$$b$cjs^bd;4I0AA@uʚ;2Nʚ;g4JdJdD:O0zppp@<4!d!dv0Pls:<4ddddv0Pls:0___PPT10 ___PPT9d\E?~- j2009 Stata Conference: Subpop Analysis of Survey DataO =HEUsing Stata for Subpopulation Analysis of Complex Sample Survey DatajBrady T. West PhD Student Michigan Program in Survey Methodology July 30, 2009 2009 Stata Conference(kA)Presentation OutlineIntroduction: Subclass Analysis Issues Kish s Taxonomy of Subclasses Two Alternative Approaches to Inference Variance Estimation and Methods for Singletons Examples using NHANES and NHAMCS Data Suggestions for Practice Directions for Future Research" Subclass Analysis IssuesAnalysts of large, complex sample survey data sets are often interested in making inferences about subpopulations of the original population that the sample was selected from (e.g., Caucasian Females) These subpopulations are referred to interchangeably in various literatures as subgroups, subclasses, subpopulations, domains, and subdomains, leading to confusion among analysts of survey datajZB ]L 5)&@Subclass Analysis Issues, cont dJSoftware procedures for analysis of complex sample survey data are becoming more powerful, flexible, and widely available, offering analysts several options Analysts need to be careful when analyzing subclasses, and be aware of the alternative approaches to subclass analysis that are possible and their implications for inference :Kish s Taxonomy of SubclassesDesign Domains: Restricted to specific strata according to the complex sample design (usually geographically, e.g., Texas) Cross-Classes: Broadly distributed (in theory) across the strata and primary sampling units defining a complex sample (e.g., African-Americans over age 50) Mixed Classes: Disproportionately distributed across the complex sample design (e.g., Hispanics in a sample including Los Angeles as a stratum) See Kish (1987), Statistical Design for ResearchZN r *(-Design DomainsX = Sample Element in Subclass..+) Cross-Classes-* Mixed Classes,'0Applying Kish s TaxonomyThe type of subclass is critical for determining an appropriate analysis approach Two possible approaches to inference motivated by the taxonomy: 1. Unconditional approach (cross-classes, mixed classes) 2. Conditional approach (design domains) pZeZZv-.+The Unconditional ApproachQAppropriate for Cross-Classes, and in some cases Mixed Classes; the subclass of interest theoretically can appear in all design strata and primary sampling units (PSUs) KEY POINT: Allow the software to process the entire survey data set, and recognize all possible design strata and PSUs; DO NOT delete sample cases not in the subclass! R k #D*,t3/,The Unconditional ApproachRationale: estimated variances for sample estimates of subclass parameters (based on within-stratum variance between PSUs) need to reflect sample-to-sample variability based on the full complex design In other words, if a particular subclass does not appear in a PSU in any given sample (although in theory it could have), that PSU should contribute 0 to variance estimates, rather than be ignored completely!VZ !u!=:The Unconditional ApproachFurther, the subclass sample size in each stratum is going to be a random variable, and theoretical sample-to-sample variance in realizations of this random variable should be incorporated into any variance estimation procedures4C0-The Unconditional ApproachIf cross-classes (or in some cases mixed classes) are being analyzed, and PSUs where the subclass does not appear (by random chance) are deleted, problems arise Some strata may appear to have only one PSU by design (preventing variance estimation unless an ad hoc approach is used) Entire design strata may be dropped, impacting variance estimates and calculations of degrees of freedomrZ YJ6EB/The Unconditional Approach: General Stata Code00(Asvy, subpop(indicator): command varlist, options indicator = an indicator variable for the subpop or an if condition, e.g., if male == 1 svy: mean, over(groupvar) svy: prop, over(groupvar) Stata drops strata* with no subpopulation observations from degrees of freedom calculations * Exercise: repeat 10 times really fastf) (4( 1.The Conditional ApproachAppropriate for Design Domains, where a subclass cannot appear outside of specific design strata The rationale behind the unconditional approach no longer applies Certain design strata should not contribute to variance estimation or calculation of degrees of freedom @ZI2/The Conditional ApproachKRestrict the analysis to only those design strata where the subclass of interest exists Variance estimates reflecting sample-to-sample variability should only be based on those design strata where the subclass can appear (unlike the unconditional approach) Subclass sample sizes in design domains are assumed to be fixed, by designLLZVFC-The Conditional Approach: General Stata Code..(svy: command varlist if (condition), options (condition) might be male == 1, or a more complex combination of conditions (e.g., male == 1 & age >= 50 & age <= 90)T j$ GDVariance Estimation Methods%All of these issues are only relevant when using Taylor Series Linearization, which is a default for variance estimation in Stata Conditional analyses are OK to perform when using replication methods, such as Balanced Repeated Replication or Jackknife Repeated Replication (Rust and Rao, 1996)HEfAd-hoc Fixes for Singleton Clusters in Stata 10.144(JStata 10.1 provides users with four ad-hoc fixes for the problem where strata are identified with only a single ultimate cluster for variance estimation in a subpopulation analysis: Report Missing Standard Errors (not really a fix) Treat Units as Certainty Units, which contribute nothing to the standard error Scale Variance using Certainty Units, which uses the average variance from each stratum with multiple PSUs for each stratum with only a single PSU Center at the Grand Mean, where the variance contribution comes from a deviation from the grand mean instead of the stratum meanP" P1$oi30Example: The NHANES Data@We first consider examples based on the NHANES II data set, collected from a nationally representative multistage probability sample of the U.S. population from 1976-1980 (oldie but a goodie) Briefly, a sample of the U.S. population was given medical examinations in an effort to assess the health of the U.S. population41Example NHANES Analysis<Analysis Subclass: African-Americans ages 50 and above (this is a cross-class of the U.S. population, which can theoretically appear in all design strata and PSUs) Analysis Objective: Estimate the mean systolic blood pressure of this subclass and an appropriate standard error See West et al. (2007) for more detailsn=Z1W=7IF4Conditional Approach:Stata Code for NHANES Analysis55(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(centered) Also singleunit(certainty), singleunit(scaled) gen b50subp = (race == 2 & ager >= 50) svy: mean bpsyst if b50subp == 1 PZ22g )MIConditional Approach: Results63Conditional Approach?This approach would not be appropriate for this particular subclass Computed standard errors would generally be biased downward, because additional sources of sample-to-sample variability are ignored when following this approach Same issues apply for analytic models Evidence that the scaled ad-hoc fix may be overly conservative!HMZwADBJG6Unconditional Approach:Stata Code for NHANES Analysis77(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) Note: choice of single unit option does not matter when following this approach! gen b50subp = (race == 2 & ager >= 50) svy, subpop(b50subp): mean bpsyst&2eIz74Unconditional Approach: Results85Unconditional Approach?8This approach would be the appropriate choice for a cross-class such as African-Americans over the age of 50 Inferences are theoretically appropriate Same idea for analytic models Results suggest that the centered and certainty ad-hoc fixes for conditional analyses are reasonable>;Example: The NHAMCS DataAnalysis Subclass: Visits to Emergency Departments (ED) by African-American men ages 60 and above (this is another cross-class of the U.S. population, which can theoretically appear in all NHAMCS design strata and PSUs) Analysis Objective: Estimate the percentage of all ED visits by members of this subclass for dizziness and/or vertigo in 2004 See West et al. (2008) for more details b^R(?<Stata Code for NHAMCS Analyses(svyset cpsum [pweight = patwt], strata(cstratm) singleunit(& ) generate subc = (settype == 3 & sex == 2 & agecat == 5 & race == 2) svy: tabulate dizzyrfv if subc == 1, se ci percent * conditional svy, subpop(subc): tabulate dizzyrfv, se ci percent * unconditional 4Z42 KHNHAMCS Analysis Results@=NHAMCS Analysis Implications!No problems with strata having only a single ultimate cluster: ad-hoc fixes all give the same results Weighted point estimates are identical Substantially fewer design-based degrees of freedom when following the conditional approach; the full complex design will not be reflected in estimation of sample-to-sample variance (many ultimate clusters are lost) Conditional analysis assumes that each sample will be of fixed size n = 397 for variance estimation purposes; no random variance! Conditional analysis results in overly liberal inferences d"Zd:7Suggestions for PracticeConsider Kish s Taxonomy when determining an appropriate subclass analysis approach Utilize the appropriate software options for unconditional analyses when analyzing cross-classes Be careful with missing values when creating the subpopulation indicator The unconditional analysis approach generally works fine for both cases (when in doubt, use this approach)iPiA>Directions for Future ResearchMore appropriate calculation / estimation of design-based and effective degrees of freedom for sparse subclasses or mixed classes Development of analytic theory for interval estimation when working with small subclasses, which does not rely on asymptotic results B? ReferencesNKish, L. 1987. Statistical Design for Research. New York: Wiley. Rust, K. F., and J. N. K. Rao. 1996. Variance estimation for complex surveys using replication. Statistical Methods in Medical Research 5: 283 310. West, B.T., Berglund, P., and Heeringa, S.G. 2008. A Closer Examination of Subpopulation Analysis of Complex Sample Survey Data. The Stata Journal, 8(3), 1-12. West, B.T., Berglund, P., and Heeringa, S.G. 2007. Alternative Approaches to Subclass Analysis of Complex Sample Survey Data. Proceedings of the 2007 Joint Statistical Meetings. ,(Z(_>[<9Questions / Thank You!For additional questions, comments, or electronic copies of these slides or the papers, please send an email to bwest@umich.edupr9s hM1xP( L/0LDTimes New RomanPPs:ؖO0ؖ@. @n?" dd@ @@``|DR1% ' ! ("#$%&'(3)*.?$$b$cjs^bd;4I0AA@uʚ;2Nʚ;g4JdJdD:O0zppp@<4!d!dv0Pls:<4ddddv0Pls:0___PPT10 ___PPT9d\E?~- j2009 Stata Conference: Subpop Analysis of Survey DataO =HEUsing Stata for Subpopulation Analysis of Complex Sample Survey DatajBrady T. West PhD Student Michigan Program in Survey Methodology July 30, 2009 2009 Stata Conference(kA)Presentation OutlineIntroduction: Subclass Analysis Issues Kish s Taxonomy of Subclasses Two Alternative Approaches to Inference Variance Estimation and Methods for Singletons Examples using NHANES and NHAMCS Data Suggestions for Practice Directions for Future Research" Subclass Analysis IssuesAnalysts of large, complex sample survey data sets are often interested in making inferences about subpopulations of the original population that the sample was selected from (e.g., Caucasian Females) These subpopulations are referred to interchangeably in various literatures as subgroups, subclasses, subpopulations, domains, and subdomains, leading to confusion among analysts of survey datajZB ]L 5)&@Subclass Analysis Issues, cont dJSoftware procedures for analysis of complex sample survey data are becoming more powerful, flexible, and widely available, offering analysts several options Analysts need to be careful when analyzing subclasses, and be aware of the alternative approaches to subclass analysis that are possible and their implications for inference :Kish s Taxonomy of SubclassesDesign Domains: Restricted to specific strata according to the complex sample design (usually geographically, e.g., Texas) Cross-Classes: Broadly distributed (in theory) across the strata and primary sampling units defining a complex sample (e.g., African-Americans over age 50) Mixed Classes: Disproportionately distributed across the complex sample design (e.g., Hispanics in a sample including Los Angeles as a stratum) See Kish (1987), Statistical Design for ResearchZN r *(-Design DomainsX = Sample Element in Subclass..+) Cross-Classes-* Mixed Classes,'0Applying Kish s TaxonomyThe type of subclass is critical for determining an appropriate analysis approach Two possible approaches to inference motivated by the taxonomy: 1. Unconditional approach (cross-classes, mixed classes) 2. Conditional approach (design domains) pZeZZv-.+The Unconditional ApproachQAppropriate for Cross-Classes, and in some cases Mixed Classes; the subclass of interest theoretically can appear in all design strata and primary sampling units (PSUs) KEY POINT: Allow the software to process the entire survey data set, and recognize all possible design strata and PSUs; DO NOT delete sample cases not in the subclass! R k #D*,t3/,The Unconditional ApproachRationale: estimated variances for sample estimates of subclass parameters (based on within-stratum variance between PSUs) need to reflect sample-to-sample variability based on the full complex design In other words, if a particular subclass does not appear in a PSU in any given sample (although in theory it could have), that PSU should contribute 0 to variance estimates, rather than be ignored completely!VZ !u!=:The Unconditional ApproachFurther, the subclass sample size in each stratum is going to be a random variable, and theoretical sample-to-sample variance in realizations of this random variable should be incorporated into any variance estimation procedures4C0-The Unconditional ApproachIf cross-classes (or in some cases mixed classes) are being analyzed, and PSUs where the subclass does not appear (by random chance) are deleted, problems arise Some strata may appear to have only one PSU by design (preventing variance estimation unless an ad hoc approach is used) Entire design strata may be dropped, impacting variance estimates and calculations of degrees of freedomrZ YJ6EB/The Unconditional Approach: General Stata Code00(Asvy, subpop(indicator): command varlist, options indicator = an indicator variable for the subpop or an if condition, e.g., if male == 1 svy: mean, over(groupvar) svy: prop, over(groupvar) Stata drops strata* with no subpopulation observations from degrees of freedom calculations * Exercise: repeat 10 times really fastf) (4( 1.The Conditional ApproachAppropriate for Design Domains, where a subclass cannot appear outside of specific design strata The rationale behind the unconditional approach no longer applies Certain design strata should not contribute to variance estimation or calculation of degrees of freedom @ZI2/The Conditional ApproachKRestrict the analysis to only those design strata where the subclass of interest exists Variance estimates reflecting sample-to-sample variability should only be based on those design strata where the subclass can appear (unlike the unconditional approach) Subclass sample sizes in design domains are assumed to be fixed, by designLLZVFC-The Conditional Approach: General Stata Code..(svy: command varlist if (condition), options (condition) might be male == 1, or a more complex combination of conditions (e.g., male == 1 & age >= 50 & age <= 90)T j$ GDVariance Estimation Methods%All of these issues are only relevant when using Taylor Series Linearization, which is a default for variance estimation in Stata Conditional analyses are OK to perform when using replication methods, such as Balanced Repeated Replication or Jackknife Repeated Replication (Rust and Rao, 1996)HEfAd-hoc Fixes for Singleton Clusters in Stata 10.144(JStata 10.1 provides users with four ad-hoc fixes for the problem where strata are identified with only a single ultimate cluster for variance estimation in a subpopulation analysis: Report Missing Standard Errors (not really a fix) Treat Units as Certainty Units, which contribute nothing to the standard error Scale Variance using Certainty Units, which uses the average variance from each stratum with multiple PSUs for each stratum with only a single PSU Center at the Grand Mean, where the variance contribution comes from a deviation from the grand mean instead of the stratum meanP" P1$oi30Example: The NHANES Data@We first consider examples based on the NHANES II data set, collected from a nationally representative multistage probability sample of the U.S. population from 1976-1980 (oldie but a goodie) Briefly, a sample of the U.S. population was given medical examinations in an effort to assess the health of the U.S. population41Example NHANES Analysis<Analysis Subclass: African-Americans ages 50 and above (this is a cross-class of the U.S. population, which can theoretically appear in all design strata and PSUs) Analysis Objective: Estimate the mean systolic blood pressure of this subclass and an appropriate standard error See West et al. (2007) for more detailsn=Z1W=7IF4Conditional Approach:Stata Code for NHANES Analysis55(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(centered) Also singleunit(certainty), singleunit(scaled) gen b50subp = (race == 2 & ager >= 50) svy: mean bpsyst if b50subp == 1 PZ22g )MIConditional Approach: Results63Conditional Approach?This approach would not be appropriate for this particular subclass Computed standard errors would generally be biased downward, because additional sources of sample-to-sample variability are ignored when following this approach Same issues apply for analytic models Evidence that the scaled ad-hoc fix may be overly conservative!HMZwADBJG6Unconditional Approach:Stata Code for NHANES Analysis77(svyset ppsu [pweight = fwgtexam], strata(stratum) singleunit(missing) Note: choice of single unit option does not matter when following this approach! gen b50subp = (race == 2 & ager >= 50) svy, subpop(b50subp): mean bpsyst&2eIz74Unconditional Approach: Results85Unconditional Approach?8This approach would be the appropriate choice for a cross-class such as African-Americans over the age of 50 Inferences are theoretically appropriate Same idea for analytic models Results suggest that the centered and certainty ad-hoc fixes for conditional analyses are reasonable>;Example: The NHAMCS DataAnalysis Subclass: Visits to Emergency Departments (ED) by African-American men ages 60 and above (this is another cross-class of the U.S. population, which can theoretically appear in all NHAMCS design strata and PSUs) Analysis Objective: Estimate the percentage of all ED visits by members of this subclass for dizziness and/or vertigo in 2004 See West et al. (2008) for more details b^R(?<Stata Code for NHAMCS Analyses(svyset cpsum [pweight = patwt], strata(cstratm) singleunit(& ) generate subc = (settype == 3 & sex == 2 & agecat == 5 & race == 2) svy: tabulate dizzyrfv if subc == 1, se ci percent * conditional svy, subpop(subc): tabulate dizzyrfv, se ci percent * unconditional 4Z42 KHNHAMCS Analysis Results@=NHAMCS Analysis Implications!No problems with strata having only a single ultimate cluster: ad-hoc fixes all give the same results Weighted point estimates are identical Substantially fewer design-based degrees of freedom when following the conditional approach; the full complex design will not be reflected in estimation of sample-to-sample variance (many ultimate clusters are lost) Conditional analysis assumes that each sample will be of fixed size n = 397 for variance estimation purposes; no random variance! Conditional analysis results in overly liberal inferences d"Zd:7Suggestions for PracticeConsider Kish s Taxonomy when determining an appropriate subclass analysis approach Utilize the appropriate software options for unconditional analyses when analyzing cross-classes Be careful with missing values when creating the subpopulation indicator The unconditional analysis approach generally works fine for both cases (when in doubt, use this approach)iPiA>Directions for Future ResearchMore appropriate calculation / estimation of design-based and effective degrees of freedom for sparse subclasses or mixed classes Development of analytic theory for interval estimation when working with small subclasses, which does not rely on asymptotic results B? ReferencesNKish, L. 1987. Statistical Design for Research. New York: Wiley. Rust, K. F., and J. N. K. Rao. 1996. Variance estimation for complex surveys using replication. Statistical Methods in Medical Research 5: 283 310. West, B.T., Berglund, P., and Heeringa, S.G. 2008. A Closer Examination of Subpopulation Analysis of Complex Sample Survey Data. The Stata Journal, 8(3), 1-12. West, B.T., Berglund, P., and Heeringa, S.G. 2007. Alternative Approaches to Subclass Analysis of Complex Sample Survey Data. Proceedings of the 2007 Joint Statistical Meetings. ,(Z(_>[<9Questions / Thank You!For additional questions, comments, or electronic copies of these slides or the papers, please send an email to bwest@umich.edupr@s [M1Root EntrydO)">Pictures4ICurrent User/SummaryInformation(6 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|} or Future ResearchReferencesQuestions / Thank You!Fonts UsedDesign Template Slide Titles#_[Cstatastata