Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: RE: RE: binary-decimal precision


From   "David Harrison" <[email protected]>
To   <[email protected]>
Subject   st: RE: RE: binary-decimal precision
Date   Thu, 3 Feb 2005 09:07:47 -0000

While it is clearly irritating, I would have to say that Stata has done exactly what you asked it (knowing the problems of precision).

It has taken the number .29999999999999999 (the closest it can get to .3), divided it by .10000000000000001 (the closest it can get to .1) and decided there is a remainder of .099999999999999978.

You say that this result is "quite far from zero", but I would have to disagree- if we are working modulo .1, this is only 2.77556E-17 away from being zero, just in the wrong direction.

The suggestion of using mod(float(.3),float(.1)) appears to work here, but this is just a fluke: .10000000149011612 happens to divide into .30000001192092896 just over 3 times instead of just under. Using this float approach could lead to even bigger errors, e.g.

. di %20.0g mod(float(.7),float(.1))
 .099999979138374329

The advice is clear, be very careful with mod() - and with int(), floor(), ceil() - and use integers wherever possible (e.g. store lengths in mm not m).

David
[email protected]


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index