Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

# st: RE: RE: How easy to get caught by rounding error

 From Nick Cox To "'statalist@hsphsun2.harvard.edu'" Subject st: RE: RE: How easy to get caught by rounding error Date Tue, 14 Dec 2010 10:45:46 +0000

```Not disagreeing, just amplifying slightly:

The division takes place first and 0.7 does not have an exact binary equivalent.

Although a little thought shows that the calculation reduces to 7 * 18 and should yield an integer, you get what you asked for in either case, Stata's best stab at what you asked it to calculate.

Nick
n.j.cox@durham.ac.uk

Keith Dear

Yes that is worrying:

. di %20.16f 7*180/10
126.0000000000000000

. di %20.16f 7/10*180
125.9999999999999900

. di .7*180
126

. di floor(.7*180)	// as per -help binomialtail-
125

Can't be too careful it seems!
But I have to wonder WHY the function uses floor() -- it seeems to lay a
trap without adding any useful functionality.
Safer would be to issue an error if the supposedly integer arguments
aren't. Is there a major downside to this?
Anyway, thanks for the warning.

Allan Reese

Tabulating some binomial tail probabilities, I was startled that they
were not monotonic.  To cut a long story short, the erroneous result was

binomialtail( 180 , .7*180 , .75), which was different from
binomialtail( 180 , 126 , .75),
although .7*180 insisted on printing as 126

binomialtail( 180 , 180/10*7 , .75) did equal binomialtail( 180, 126 ,
.75) as did binomialtail( 180, round(.7*180) , .75)

But
binomialtail( 180 , .7/180 , .75) = binomialtail( 180 , 125 , .75)

It's made me think how hard it is to *prove* that a computer program is
correct.  round() should fix this one - for all values?

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```