Search
   >> Home >> Resources & support >> Certification results

Diehard test results on Stata's random-number generator

  1. Explanation
  2. Summary
  3. Command and log files
  4. Diehard test p-values

1. Explanation

The Stata pseudorandom-number generator was subjected to the Diehard suite of tests, which were developed by George Marsaglia and provide extensive testing of pseudorandom-number generators. The results of those tests are presented here.

2. Summary

The distribution of the p-values from the Diehard suite of tests is shown below and indicates that Stata’s random-number generator is doing a great job.

                        Observed   "Expected"
        p-value range   Percent     Percent
        -------------   --------   ----------
         0.0 -- 0.1        10          10
         0.1 -- 0.2        12          10
         0.2 -- 0.3        10          10
         0.3 -- 0.4        10          10
         0.4 -- 0.5         9          10
         0.5 -- 0.6        12          10
         0.6 -- 0.7        11          10
         0.7 -- 0.8         7          10
         0.8 -- 0.9        11          10
         0.9 -- 1.0         9          10

When interpreting the p-values produced by Diehard, be aware of the following note at the top of the Diehard test log file—randnumb.out.

Most of the tests in Diehard return a p-value, which should be uniform on [0,1) if the input file contains truly independent random bits. Those p-values are obtained by p=F(X), where F is the assumed distribution of the sample random variable X—often normal. But that assumed F is just an asymptotic approximation, for which the fit will be worst in the tails. Thus you should not be surprised with occasional p-values near 0 or 1, such as .0012 or .9983. When a bit stream really FAILS BIG, you will get p's of 0 or 1 to six or more places. By all means, do not, as a Statistician might, think that a p < .025 or p > .975 means that the RNG has "failed the test at the .05 level". Such p's happen among the hundreds that Diehard produces, even with good RNG's. So keep in mind that "p happens".

The smallest p-value produced was .0042, and the largest was .9997, which seems reasonable given the quote above.

3. Command and log files

The web address for the Diehard suite of tests is http://stat.fsu.edu/pub/diehard/.

The Diehard suite of tests require as input a specially formatted binary file containing 3 million 32-bit integers. This specially formatted binary file was first produced, and then the Diehard tests were performed on the resulting file. Here are the steps that generated the results:

  1. The randnumb.do do-file was run within Stata Version 13 on June 17, 2013. This produced the ASCII file randnumb.raw containing 3 million 32-bit integers. randnumb.log is the corresponding Stata log file. (The same numbers are produced by Stata regardless of which operating system is used.)
  2. The C file chnghex.c was compiled and run by typing chnghex randnumb.raw randnumb.hex. This read the 3 million ASCII numbers from randnumb.raw and produced the ASCII file randnumb.hex containing the hexadecimal representation of the 3 million numbers.
  3. The Diehard program asc2bin was run on an Intel i5-based computer running Windows 8. This program requested a hexadecimal ASCII input file (randnumb.hex) and produced a Diehard specially formatted binary file (randnumb.bin).
  4. The Diehard program diehard was run on an Intel i5-based computer running Windows 8. The program requested a specially formatted binary file (randnumb.bin) and that all tests be performed. The program then performed the tests and produced a log file. That file, randnumb.out, was then copied to this website.

4. Diehard test p-values

The following table shows the p-values produced by the Diehard suite of tests. Click here to see the full Diehard test log file from which these numbers were extracted. A summary of the distribution of these p-values is shown above, along with a note on how to interpret these p-values.

        +---------------------------------------------------------------------------+
        | Birthday Spacing | OPSO             | Count 1s Stream  | Overlapping Sum  |
        | ---------------- | ---------------- | ---------------- | ---------------- |
        | .049660          | .6201            | .034771          | .687281          |
        | .685700          | .8671            | .110043          | .239832          |
        | .320627          | .1438            |                  | .780777          |
        | .339908          | .8054            |                  | .686973          |
        | .682586          | .8110            | Count 1s Bytes   | .658549          |
        | .790704          | .1856            | ---------------- | .914380          |
        | .254279          | .3118            | .610413          | .266535          |
        | .058541          | .3069            | .440286          | .836885          |
        | .121231          | .1122            | .874250          | .603915          |
        | ---------------- | .8611            | .299979          | .289208          |
        | .749450  KSTEST  | .5188            | .122604          | ---------------- |
        |                  | .9447            | .868545          | .635848  KSTEST  |
        |                  | .4968            | .512628          |                  |
        | OPERM5           | .4216            | .989226          |                  |
        | ---------------- | .3094            | .553270          | Runs             |
        | .043421          | .5869            | .168868          | ---------------- |
        | .381748          | .2320            | .561769          | .698717          |
        |                  | .4284            | .258610          | .628001          |
        |                  | .4136            | .936716          | .304695          |
        | Bin. Rank 31x31  | .5325            | .815773          | .254565          |
        | ---------------- | .8618            | .999685          |                  |
        | .322191          | .6689            | .191903          |                  |
        |                  | .4748            | .030241          | Craps Wins       |
        |                  |                  | .649521          | ---------------- |
        | Bin. Rank 32x32  |                  | .491447          | .050885          |
        | ---------------- | OQSO             | .275806          |                  |
        | .329856          | ---------------- | .955717          |                  |
        |                  | .3330            | .655291          | Craps Throws     |
        |                  | .1551            | .817905          | ---------------- |
        | Bin. Rank 6x8    | .9847            | .159428          | .116953          |
        | ---------------- | .5960            | .667430          |                  |
        | .492591          | .4177            |                  |                  |
        | .140335          | .6297            |                  |                  |
        | .552921          | .0042            | CDPARK           |                  |
        | .363156          | .5023            | ---------------- |                  |
        | .635619          | .7868            | .753306          |                  |
        | .848542          | .3823            | .205562          |                  |
        | .150218          | .4296            | .323972          |                  |
        | .648319          | .1028            | .907282          |                  |
        | .483047          | .4216            | .261324          |                  |
        | .552624          | .3404            | .006836          |                  |
        | .864940          | .9997            | .853193          |                  |
        | .025812          | .9853            | .642555          |                  |
        | .432879          | .5360            | .276387          |                  |
        | .850342          | .5023            | .518210          |                  |
        | .774656          | .3784            | ---------------- |                  |
        | .060820          | .8411            | .091924  KSTEST  |                  |
        | .830375          | .1625            |                  |                  |
        | .268389          | .3733            |                  |                  |
        | .410072          | .1264            | Minimum Dist.    |                  |
        | .964345          | .1358            | ---------------- |                  |
        | .995751          | .0493            | .690770          |                  |
        | .791425          | .2957            |                  |                  |
        | .886268          | .1797            |                  |                  |
        | .006625          | .5602            | 3D Spheres       |                  |
        | .706425          |                  | ---------------- |                  |
        | ---------------- |                  | .00667           |                  |
        | .459437  KSTEST  | DNA              | .49696           |                  |
        |                  | ---------------- | .98401           |                  |
        |                  | .0985            | .24728           |                  |
        | Overlap 20       | .5126            | .99245           |                  |
        | ---------------- | .9932            | .07170           |                  |
        | .12262           | .5837            | .80567           |                  |
        | .85059           | .3150            | .53561           |                  |
        | .49969           | .7091            | .42710           |                  |
        | .22710           | .7131            | .55683           |                  |
        | .13540           | .9574            | .97005           |                  |
        | .17541           | .6235            | .89231           |                  |
        | .13540           | .1902            | .07843           |                  |
        | .15791           | .3139            | .71144           |                  |
        | .96375           | .5906            | .53868           |                  |
        | .70100           | .7121            | .57628           |                  |
        | .20787           | .2204            | .59247           |                  |
        | .08944           | .8003            | .85392           |                  |
        | .51181           | .3129            | .52476           |                  |
        | .90725           | .1831            | .66157           |                  |
        | .37948           | .0761            | ---------------- |                  |
        | .65755           | .2475            | .775862  KSTEST  |                  |
        | .47641           | .3646            |                  |                  |
        | .74402           | .2186            |                  |                  |
        | .03344           | .2569            | SQUEEZE          |                  |
        | .58935           | .9503            | ---------------- |                  |
        |                  | .6235            | .843286          |                  |
        |                  | .1815            |                  |                  |
        |                  | .0237            |                  |                  |
        |                  | .7040            |                  |                  |
        |                  | .4305            |                  |                  |
        |                  | .2041            |                  |                  |
        |                  | .5466            |                  |                  |
        |                  | .8187            |                  |                  |
        +---------------------------------------------------------------------------+
The Stata Blog: Not Elsewhere Classified Find us on Facebook Follow us on Twitter LinkedIn Google+ Watch us on YouTube