Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: RE: How easy to get caught by rounding error


From   "Keith Dear" <[email protected]>
To   <[email protected]>
Subject   st: RE: How easy to get caught by rounding error
Date   Tue, 14 Dec 2010 09:54:25 +1100

Yes that is worrying:

. di %20.16f 7*180/10
126.0000000000000000

. di %20.16f 7/10*180
125.9999999999999900

. di .7*180
126

. di floor(.7*180)	// as per -help binomialtail-
125

Can't be too careful it seems! 
But I have to wonder WHY the function uses floor() -- it seeems to lay a
trap without adding any useful functionality.
Safer would be to issue an error if the supposedly integer arguments
aren't. Is there a major downside to this?
Anyway, thanks for the warning.
kd



-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Allan Reese
(Cefas)
Sent: Tuesday, 14 December 2010 4:29 AM
To: [email protected]
Cc: David Smith (Cefas); David Maxwell (Cefas); Jon Barry (Cefas); Mark
Thrush (Cefas); Nick Taylor (Cefas)
Subject: st: How easy to get caught by rounding error

Tabulating some binomial tail probabilities, I was startled that they
were not monotonic.  To cut a long story short, the erroneous result was


binomialtail( 180 , .7*180 , .75), which was different from
binomialtail( 180 , 126 , .75), 
although .7*180 insisted on printing as 126 

binomialtail( 180 , 180/10*7 , .75) did equal binomialtail( 180, 126 ,
.75) as did binomialtail( 180, round(.7*180) , .75) 

But 
binomialtail( 180 , .7/180 , .75) = binomialtail( 180 , 125 , .75)

It's made me think how hard it is to *prove* that a computer program is
correct.  round() should fix this one - for all values?

Allan


R Allan Reese
Senior statistician, Cefas
The Nothe, Weymouth DT4 8UB 

Tel: +44 (0)1305 206614 -direct
Fax: +44 (0)1305 206601 

www.cefas.co.uk 




************************************************************************
***********
This email and any attachments are intended for the named recipient
only.  Its unauthorised use, distribution, disclosure, storage or
copying is not permitted.  If you have received it in error, please
destroy all copies and notify the sender.  In messages of a non-business
nature, the views and opinions expressed are the author's own and do not
necessarily reflect those of the organisation from which it is sent.
All emails may be subject to monitoring.
************************************************************************
***********


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index