Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: st: Question about floating point precision

From   "Nick Cox" <>
To   <>
Subject   RE: st: Question about floating point precision
Date   Tue, 30 Mar 2010 13:49:33 +0100

Phil's good answer can be supplemented. 

The underlying principle, as Markus knows well, is that what look like
exact decimals don't necessarily correspond to exact binaries. In fact,
sometimes you get exactly the right result, sometimes a smidgen too
much, and sometimes a smidgen too little.  

In this particular case, let's follow the point made clear by Phil. The
format %21x lets you drill down a little deeper to see what is

. di %21x  26.07

. di %21x  52.14

shows that luck, or caprice, or the numbers, can be on Markus' side.
52.14 happens to be held internally as _exactly_ twice what 26.07 is
held as, so the ratio comes out as 0.5 and Markus gets exactly the right
answer in his first example. That's good fortune not matched in the
other ways of holding numbers. 


Phil Schumm

On Mar 30, 2010, at 12:37 AM, Markus Hahn wrote:
> ----------------------------------
> . // why is this code more precise
> .
> . di %22.18f 1559*26.07/52.14
> 779.500000000000000000
> .
> . // than this code
> .
> . di %22.18f 1559*26.07
> 4.064313000000000e+04
> . di %22.18f 40643.13/52.14
> 779.499999999999890000
> ----------------------------------
> Why is the code in the first part of my example more precise than  
> the last part? Is it really more precise or does it only SEEM to be  
> so? What I would like to know is whether the first part is more  
> precise (if at all) because it only uses one expression for the  
> calculation as compared to last part which uses two expressions?  
> Does the indirect calculation introduce this imprecision?

The difference is due to the order of operations -- remember, / comes  
before *.  To wit:

. di %21x 1559*26.07/52.14

. di %21x (1559*26.07)/52.14

. di %21x 40643.13/52.14

*   For searches and help try:

© Copyright 1996–2016 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index