[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]
RE: st: Collapse & Missing Values
Thank you Nick for your valiant effort to characterize the treatment of missings as a feature. And thank you, Friedrich, for your work-around (& again to you Nick for your help on that too).
Let me just try to explain why this wasn't a feature for me today. Using the collapse statement, I was aggregating various amount fields by day. There could be multiple (and usually were) transactions per day. Once I had the aggregated amounts, I was interested in their correlations, especially the correlation of one amount with the lagged amount of another. When I start introducing erroneous zero amounts, my correlations will not be unbiased, & certainly not correct. In fact, the way I discovered this is that one colleague was computing the same correlations in SAS. For some reason, I had more observations than he. I now know that my "extra" observations were the result of collapse's treatment of missing values. I was able to get the same correlations as my colleague by deleting the observations with missing amounts but then I also lose the information on the number of transactions on those days (albeit with incomplete data). So yes, I emphatically agree with your d
>I guess what Eric would in effect like Stata to do
>is to keep track of all the occurrences of
>missing so that -sum()- would produce say
>. + . + . + . + . + . + 42 = 42
>. + . + . + . + . + . + . = .
>Thus, at the end of a set that were all missing,
>-sum()- would be morally compelled to say,
>"No, that initial guess of 0 doesn't apply here.
>These values are all missing, so the sum must
>be missing. I changed my mind!"
Failing such a radical change to collapse, perhaps there could be an "allmiss" parameter that would make the sum of totally missing values equal to missing.
Eric G. Wruck
2535 Sherwood Road
Columbus, OH 43209
* For searches and help try: