Re: st: Fraction 0.8 vs 0.8000001

[Maarten Buis <maartenlbuis@gmail.com>]

On Fri, Dec 23, 2011 at 4:33 PM, qing ye  wrote:
> Why when I try to generate variable=(5-1)/5, stata cannot generate a
> variable equal to 0.8, the result the code generated is 0.8000001, why
> is this happening? How can I fix it?

No (binary) computer can store that number exactly. Computers store
numbers in binary and in binary a number like .8 is like 1/3 in
decimal: it cannot be represented in a finite number of digits. As a
consequence a computer has to round those numbers. This is an issue
that is often referred to as precision on this list and there is a lot
of useful material on it:

http://blog.stata.com/2011/02/02/how-to-read-the-percent-21x-format/
http://blog.stata.com/2011/02/10/how-to-read-the-percent-21x-format-part-2/
http://blog.stata.com/2011/06/17/precision-yet-again-part-i/
http://blog.stata.com/2011/06/23/precision-yet-again-part-ii/


William Gould (2006) Mata Matters: Precision. The Stata Journal. 6(4):550-560.
http://www.stata-journal.com/article.html?article=pr0025

Nicholas J. Cox (2006) "Stata tip 33: Sweet sixteen:
Hexadecimal formats and precision problems" The Stata
Journal, 6(2): 282-283.
http://www.stata-journal.com/article.html?article=dm0022

Jean Marie Linhart (2008) "Mata matters: Overflow,
underflow and the IEEE floating-point format" The
Stata Journal, 8(2): 255-268.
http://www.stata-journal.com/article.html?article=pr0038

http://www.stata.com/support/faqs/data/prec.html

http://www.stata.com/support/faqs/data/float.html

http://www.ats.ucla.edu/stat/stata/faq/longid.htm

http://www.michaelnormanmitchell.com/stow/floats-and-doubles.html

Hope this helps,
Maarten

--------------------------
Maarten L. Buis
Institut fuer Soziologie
Universitaet Tuebingen
Wilhelmstrasse 36
72074 Tuebingen
Germany


http://www.maartenbuis.nl
--------------------------
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/