st: RE: binary-decimal precision

 From "Nick Cox" To Subject st: RE: binary-decimal precision Date Wed, 2 Feb 2005 20:58:16 -0000

```Chris's initial question looks like one for Microsoft.

Apart from noting a dubious assumption that Stata users
are refugees from Excel -- many never lived in that land --
I want to make three additional points on various levels:

1. Stata pays users the compliment of giving them what
they asked for, as best it can. It turns out that
this isn't always popular. And all of us can fail, at
least momentarily, to see the implications of what we

2. That return is not always what users expect,
but Stata doesn't have ways of reaching into users' minds
and changing their expectation, except via the documentation.

3. -mod(0.3,0.1)- is, in a way, an embarrassment. Any human knowing
the definition of -mod(0.3,0.1)- knows that the answer should be 0.

_But_ how to put that in the software is the big issue.

I guess that the code that does the calculation never really "sees" the
0.3 or 0.1 -- and it certainly doesn't do the pattern recognition of
saying "hey, the ratio is an integer, so the answer will be 0".
There could be a (partial?) fix in terms of writing
all sorts of extra functions guaranteed to give integer results,
but at the expense of bloating both code and documentation.

The point is also made by the cube root example aired some
while on Statalist.

.  di 8^(1/3)
2

.  di %21x 8^(1/3)
+1.fffffffffffffX+000

.  di %21x 2
+1.0000000000000X+001

shows that Stata doesn't (indeed can't) get it exactly right.

Nick
n.j.cox@durham.ac.uk

Chris Ruebeck (modulo HTML, stripped here)

On Feb 2, 2005, at 8:51 AM, Nick Cox wrote:

This is the old binary-decimal precision issue again. Most multiples of 0.1 cannot be held exactly as binary numbers.

After having the math involved in this issue explained in the past, I have continued to wonder why Microsoft Excel doesn't exhibit this "feature" of binary arithmetic being represented in decimal form. What do we give up in Excel in order to avoid it? Or, more correctly, why do we observe it less frequently in Excel than in Stata?

The most succinct answer I could find from a very quick search of the statalist archives ("binary-decimal precision") is quoted at the end of this message: in Excel, we give up storage space in return for avoiding the puzzle. More material is found in the following links---and, of course, in [U] 16.10 "Precision and problems therein."

http://www.stata.com/statalist/archive/2003-09/msg00234.html
http://www.stata.com/statalist/archive/2002-12/msg00490.html
http://www.stata.com/statalist/archive/2005-01/msg00318.html
http://www.stata.com/support/faqs/data/mod.html

But I would like to understand the issue better. So, taking the example in the last link to heart, I fired up Excel and entered =mod(0.3,0.1) to find that Excel does not answer 0 either, but instead -2.77556E-17. Yet this is much closer to zero than the 0.1 result that Stata provides when told to -display mod(0.3,0.1)-. Note that -di %21.18f mod(.3,.1)- produces 0.099999999999999978, which is still quite far from zero (and leads to a standard output rounded to 0.1). Note, too, that it seems a ROUND function might more consistently solve the issue in Excel than in Stata. I leave it to others to draw potential connections between Excel's treatment and the string solutions in Stata.

It's certainly possible that there is a mistake in my analysis, I don't mean that Excel should be our benchmark for statistics, and I apologize for continuing to beat this old horse.

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```