# Re: st: RE: RE: binary-decimal precision

 From Chris Ruebeck To statalist@hsphsun2.harvard.edu Subject Re: st: RE: RE: binary-decimal precision Date Thu, 3 Feb 2005 08:17:19 -0500

direction. It just looks closer to zero because -2.77556E-17 is also close to 0 when we're not using modulo 0.1 arithmetic, and so more obviously rounds to 0---which is 0.1.

So we ten-fingered creatures should internalize both the fact that computers are binary and that mod(0.1,0.1) = mod(0,0.1) = 0. (David already said this.)

It's also more obvious now how to write code to deal with this: compare both to 0 and to 0.1. If it's "close enough" (within some number bigger than machine precision) of either, then it's 0.

I continue to like Excel's answer better---whether I'm accused of refugee status or not! Yes, we sacrifice speed if we ask Stata/Excel to take care of making it "look" close to 0 instead of 0.1, but we may benefit by having fewer errors in our code, and less time debugging when we catch the errors. This is a special case of a general principle.

But I am truly satisfied with these answers now, feeling that I really understand what is happening (regardless of my opinion on the level to which human-machine interaction should be taken into account in the design of machines).

Thanks for indulging me,
Chris

On Feb 3, 2005, at 4:07 AM, David Harrison wrote:

While it is clearly irritating, I would have to say that Stata has done exactly what you asked it (knowing the problems of precision).

It has taken the number .29999999999999999 (the closest it can get to .3), divided it by .10000000000000001 (the closest it can get to .1) and decided there is a remainder of .099999999999999978.

You say that this result is "quite far from zero", but I would have to disagree- if we are working modulo .1, this is only 2.77556E-17 away from being zero, just in the wrong direction.

The suggestion of using mod(float(.3),float(.1)) appears to work here, but this is just a fluke: .10000000149011612 happens to divide into .30000001192092896 just over 3 times instead of just under. Using this float approach could lead to even bigger errors, e.g.

. di %20.0g mod(float(.7),float(.1))
.099999979138374329

The advice is clear, be very careful with mod() - and with int(), floor(), ceil() - and use integers wherever possible (e.g. store lengths in mm not m).

David
david@icnarc.org
```*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```