Diehard test results on Stata's random-number generator
- Explanation
- Summary
- Command and log files
- Diehard test p-values
1. Explanation
The Stata pseudorandom-number generator was subjected to the Diehard suite
of tests, which were developed by George Marsaglia and provide extensive
testing of pseudorandom-number generators. The results of those tests are
presented here.
2. Summary
The distribution of the p-values from the Diehard suite of tests is shown
below and indicates that Stata’s random-number generator is doing a
great job.
Observed "Expected"
p-value range Percent Percent
------------- -------- ----------
0.0 -- 0.1 10 10
0.1 -- 0.2 12 10
0.2 -- 0.3 10 10
0.3 -- 0.4 10 10
0.4 -- 0.5 9 10
0.5 -- 0.6 12 10
0.6 -- 0.7 11 10
0.7 -- 0.8 7 10
0.8 -- 0.9 11 10
0.9 -- 1.0 9 10
When interpreting the p-values produced by Diehard, be aware of the
following note at the top of the Diehard test log
file—randnumb.out.
Most of the tests in Diehard return a p-value, which should be uniform on
[0,1) if the input file contains truly independent random bits. Those
p-values are obtained by p=F(X), where F is the assumed distribution of the
sample random variable X—often normal. But that assumed F is just an
asymptotic approximation, for which the fit will be worst in the tails. Thus
you should not be surprised with occasional p-values near 0 or 1, such as
.0012 or .9983. When a bit stream really FAILS BIG, you will get p's of 0
or 1 to six or more places. By all means, do not, as a Statistician might,
think that a p < .025 or p > .975 means that the RNG has "failed the
test at the .05 level". Such p's happen among the hundreds that Diehard
produces, even with good RNG's. So keep in mind that "p happens".
The smallest p-value produced was .0042, and the largest was .9997, which
seems reasonable given the quote above.
3. Command and log files
The web address for the Diehard suite of tests is
http://stat.fsu.edu/pub/diehard/.
The Diehard suite of tests require as input a specially formatted binary
file containing 3 million 32-bit integers. This specially formatted binary
file was first produced, and then the Diehard tests were performed on the
resulting file. Here are the steps that generated the results:
- The randnumb.do do-file was
run within Stata Version 6.0 on 11 August 1999. This produced the ASCII
file randnumb.raw containing 3 million 32-bit integers. randnumb.log is the
corresponding Stata log file. (The same numbers are produced by Stata
regardless of which operating system is used.)
- The C file chnghex.c was
compiled and run by typing chnghex randnumb.raw randnumb.hex.
This read the 3 million ASCII numbers from randnumb.raw and
produced the ASCII file randnumb.hex containing the hexadecimal
representation of the 3 million numbers.
- The Diehard program asc2bin
was run on a Pentium processor computer running Windows 98. This program
requested a hexadecimal ASCII input file (randnumb.hex) and
produced a Diehard specially formatted binary file (randnumb.bin).
- The Diehard program diehard
was run on a Pentium processor computer running Windows 98. The program
requested a specially formatted binary file (randnumb.bin) and
that all tests be performed. The program then performed the tests and
produced a log file. That file,
randnumb.out, was then copied to this website.
4. Diehard test p-values
The following table shows the p-values produced by the Diehard suite of tests.
Click here to see the full Diehard test log file
from which these numbers were extracted. A summary of the distribution of
these p-values is shown above, along with a note on
how to interpret these p-values.
+---------------------------------------------------------------------------+
| Birthday Spacing | OPSO | Count 1s Stream | Overlapping Sum |
| ---------------- | ---------------- | ---------------- | ---------------- |
| .049660 | .6201 | .034771 | .687281 |
| .685700 | .8671 | .110043 | .239832 |
| .320627 | .1438 | | .780777 |
| .339908 | .8054 | | .686973 |
| .682586 | .8110 | Count 1s Bytes | .658549 |
| .790704 | .1856 | ---------------- | .914380 |
| .254279 | .3118 | .610413 | .266535 |
| .058541 | .3069 | .440286 | .836885 |
| .121231 | .1122 | .874250 | .603915 |
| ---------------- | .8611 | .299979 | .289208 |
| .749450 KSTEST | .5188 | .122604 | ---------------- |
| | .9447 | .868545 | .635848 KSTEST |
| | .4968 | .512628 | |
| OPERM5 | .4216 | .989226 | |
| ---------------- | .3094 | .553270 | Runs |
| .043421 | .5869 | .168868 | ---------------- |
| .381748 | .2320 | .561769 | .698717 |
| | .4284 | .258610 | .628001 |
| | .4136 | .936716 | .304695 |
| Bin. Rank 31x31 | .5325 | .815773 | .254565 |
| ---------------- | .8618 | .999685 | |
| .322191 | .6689 | .191903 | |
| | .4748 | .030241 | Craps Wins |
| | | .649521 | ---------------- |
| Bin. Rank 32x32 | | .491447 | .050885 |
| ---------------- | OQSO | .275806 | |
| .329856 | ---------------- | .955717 | |
| | .3330 | .655291 | Craps Throws |
| | .1551 | .817905 | ---------------- |
| Bin. Rank 6x8 | .9847 | .159428 | .116953 |
| ---------------- | .5960 | .667430 | |
| .492591 | .4177 | | |
| .140335 | .6297 | | |
| .552921 | .0042 | CDPARK | |
| .363156 | .5023 | ---------------- | |
| .635619 | .7868 | .753306 | |
| .848542 | .3823 | .205562 | |
| .150218 | .4296 | .323972 | |
| .648319 | .1028 | .907282 | |
| .483047 | .4216 | .261324 | |
| .552624 | .3404 | .006836 | |
| .864940 | .9997 | .853193 | |
| .025812 | .9853 | .642555 | |
| .432879 | .5360 | .276387 | |
| .850342 | .5023 | .518210 | |
| .774656 | .3784 | ---------------- | |
| .060820 | .8411 | .091924 KSTEST | |
| .830375 | .1625 | | |
| .268389 | .3733 | | |
| .410072 | .1264 | Minimum Dist. | |
| .964345 | .1358 | ---------------- | |
| .995751 | .0493 | .690770 | |
| .791425 | .2957 | | |
| .886268 | .1797 | | |
| .006625 | .5602 | 3D Spheres | |
| .706425 | | ---------------- | |
| ---------------- | | .00667 | |
| .459437 KSTEST | DNA | .49696 | |
| | ---------------- | .98401 | |
| | .0985 | .24728 | |
| Overlap 20 | .5126 | .99245 | |
| ---------------- | .9932 | .07170 | |
| .12262 | .5837 | .80567 | |
| .85059 | .3150 | .53561 | |
| .49969 | .7091 | .42710 | |
| .22710 | .7131 | .55683 | |
| .13540 | .9574 | .97005 | |
| .17541 | .6235 | .89231 | |
| .13540 | .1902 | .07843 | |
| .15791 | .3139 | .71144 | |
| .96375 | .5906 | .53868 | |
| .70100 | .7121 | .57628 | |
| .20787 | .2204 | .59247 | |
| .08944 | .8003 | .85392 | |
| .51181 | .3129 | .52476 | |
| .90725 | .1831 | .66157 | |
| .37948 | .0761 | ---------------- | |
| .65755 | .2475 | .775862 KSTEST | |
| .47641 | .3646 | | |
| .74402 | .2186 | | |
| .03344 | .2569 | SQUEEZE | |
| .58935 | .9503 | ---------------- | |
| | .6235 | .843286 | |
| | .1815 | | |
| | .0237 | | |
| | .7040 | | |
| | .4305 | | |
| | .2041 | | |
| | .5466 | | |
| | .8187 | | |
+---------------------------------------------------------------------------+
|
Resources & support
FAQs
Technical services
NetCourses
Short courses
Users Group meetings
Statalist
Links
Software updates
Software archives
Customer service
Manuals & supplements
Stata Journal
Stata News
Stata Automation
Plugins
|
