[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]
st: RE: RE: binary-decimal precision
While it is clearly irritating, I would have to say that Stata has done exactly what you asked it (knowing the problems of precision).
It has taken the number .29999999999999999 (the closest it can get to .3), divided it by .10000000000000001 (the closest it can get to .1) and decided there is a remainder of .099999999999999978.
You say that this result is "quite far from zero", but I would have to disagree- if we are working modulo .1, this is only 2.77556E-17 away from being zero, just in the wrong direction.
The suggestion of using mod(float(.3),float(.1)) appears to work here, but this is just a fluke: .10000000149011612 happens to divide into .30000001192092896 just over 3 times instead of just under. Using this float approach could lead to even bigger errors, e.g.
. di %20.0g mod(float(.7),float(.1))
The advice is clear, be very careful with mod() - and with int(), floor(), ceil() - and use integers wherever possible (e.g. store lengths in mm not m).
* For searches and help try: