Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Question about floating point precision


From   Phil Schumm <pschumm@uchicago.edu>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Question about floating point precision
Date   Tue, 30 Mar 2010 05:04:01 -0500

On Mar 30, 2010, at 12:37 AM, Markus Hahn wrote:
----------------------------------
. // why is this code more precise
.
. di %22.18f 1559*26.07/52.14
779.500000000000000000

.
. // than this code
.
. di %22.18f 1559*26.07
4.064313000000000e+04

. di %22.18f 40643.13/52.14
779.499999999999890000
----------------------------------

Why is the code in the first part of my example more precise than the last part? Is it really more precise or does it only SEEM to be so? What I would like to know is whether the first part is more precise (if at all) because it only uses one expression for the calculation as compared to last part which uses two expressions? Does the indirect calculation introduce this imprecision?


The difference is due to the order of operations -- remember, / comes before *. To wit:

. di %21x 1559*26.07/52.14
+1.85c0000000000X+009

. di %21x (1559*26.07)/52.14
+1.85bffffffffffX+009

. di %21x 40643.13/52.14
+1.85bffffffffffX+009


-- Phil

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index