Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: Cautionary tale - single & double


From   "Allan Reese (Cefas)" <allan.reese@cefas.co.uk>
To   "Allan Reese (Cefas)" <allan.reese@cefas.co.uk>, "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Subject   st: Cautionary tale - single & double
Date   Mon, 21 Oct 2013 13:20:51 +0000

It's well know that computers work to finite precision, but less obvious how you might detect when this breaks down.  In this example, a column of values copied from an Excel sheet gave a different total in Stata from that shown in Excel. The -table- command, however, gave the same total as Excel.  A difference of 13 in 740 million was just irritating!

The explanation was that the column of values read from Excel had been coerced to double precision, but the generated running total used single precision arithmetic.  This was further obscured by applying a format with no decimal places.

Here's the stripped down example, with the column summed in both single and double precision. The difference becomes meaningful after the first big amount is added in, but it is odd that the sign of diff switches.
Allan

* column -xlvals- read from spreadsheet

. format  xlvals  %15.3f
. gen singletot = sum(xlvals)
. g double doubletot = sum(xlvals)
. g diff = doubletot - singletot
. format single  double   %13.0f


. desc

Contains data
  obs:            33                          
 vars:             4                          
 size:           792                          
-------------------------------------------------------------
              storage  display     value
variable name   type   format      label      variable label
-------------------------------------------------------------
xlvals          double %15.3f               
singletot       float  %13.0f                 
doubletot       double %13.0f                 
diff            float  %9.0g                  

. list

     +---------------------------------------------------+
     |        xlvals   singletot   doubletot        diff |
     |---------------------------------------------------|
  1. |     69700.461       69700       69700    -.000108 |
  2. |      9504.608       79205       79205   -.0011881 |
  3. |    134792.627      213998      213998   -.0072725 |
  4. |      1036.866      215035      215035    -.000288 |
  5. |   1392857.143     1607892     1607892   -.0449309 |
     |---------------------------------------------------|
  6. |    673963.134     2281855     2281855    .0887097 |
  7. |    347350.230     2629205     2629205    .0691244 |
  8. |   1241474.654     3870680     3870680   -.0264977 |
  9. |  24436866.359    28307546    28307546    .0829493 |
 10. |    893087.558    29200634    29200634    -.359447 |
     |---------------------------------------------------|
 11. |   5073732.719    34274368    34274366   -1.640553 |
 12. |     50691.244    34325056    34325058    1.603687 |
 13. |     31105.991    34356164    34356164   -.4055299 |
 14. |    101612.903    34457776    34457776    .4976959 |
 15. |   2540322.581    36998100    36998099    -.921659 |
     |---------------------------------------------------|
 16. |    472811.060    37470912    37470910   -1.861751 |
 17. |    101612.903    37572524    37572523   -.9585254 |
 18. |     48387.097    37620912    37620910   -1.861751 |
 19. | 677845714.286   715466624   715466624    .4239631 |
 20. |     52419.355   715519040   715519044    3.778802 |
     |---------------------------------------------------|
 21. |   2730184.332   718249216   718249228     12.1106 |
 22. |    144470.046   718393728   718393698   -29.84332 |
 23. |         0.000   718393728   718393698   -29.84332 |
 24. |    892165.899   719285888   719285864    -23.9447 |
 25. |    201612.903   719487488   719487477   -11.04147 |
     |---------------------------------------------------|
 26. |  14594130.184   734081600   734081607    7.142857 |
 27. |   6007955.069   740089536   740089562    26.21198 |
 28. |     96751.152   740186304   740186313    9.364056 |
 29. |     18571.429   740204864   740204885    20.79263 |
 30. |      8064.516   740212928   740212949    21.30876 |
     |---------------------------------------------------|
 31. |     35714.286   740248640   740248664    23.59447 |
 32. |      3571.429   740252224   740252235    11.02304 |
 33. |     35714.286   740287936   740287949    13.30876 |
     +---------------------------------------------------+



*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index