Goodness-of-fit test for random generators

Quote:

Hi,

a simple question: if I have a uniform random number generator (in
[0,1]) and I do a chi^2 goodness of fit test then I would reject the
Null hypothesis if the chi^2 value is larger than chi^2_{df;1-alpha}.
Usually I would never reject the Null hypothesis if the chi^2 value is
smaller than chi^2_{df;alpha/2} or larger than chi^2_{df;1-alpha/2}.
Could be any reason for such an approach?

Thanks ins advance

Sigbert Klinke

Quote:

Sigbert Klinke <sigbert@wiwi.hu-berlin.de> wrote in news:ep1srh$q5$1
@hahn.informatik.hu-berlin.de:

Hi,

a simple question: if I have a uniform random number generator (in
[0,1]) and I do a chi^2 goodness of fit test then I would reject the
Null hypothesis if the chi^2 value is larger than chi^2_{df;1-alpha}.
Usually I would never reject the Null hypothesis if the chi^2 value is
smaller than chi^2_{df;alpha/2} or larger than chi^2_{df;1-alpha/2}.
Could be any reason for such an approach?

(A non-statistician's answer:)
Appears to me that your current approach is too narrow. If you are
testing a RNG, then you should be testing at both alpha/2 and
1-alpha/2, and maybe even other quantiles. The answer to whether to
look at both extremes depends on a proper formulation of what "should"
happen. RNGs should not be "too close" or "too far" from the
theoretical result.

I am not convinced that this would be a general answer to the question
of whether to reject the null if you get a nearly predicted result and
are using a chi-square test, but I think it is fair when you are doing
simulations which have a well-characterized mathematical form.

Quote:

On 2007-01-22 11:10:42 -0400, David Winsemius <doe_snot@comcast.n0T> said:

Sigbert Klinke <sigbert@wiwi.hu-berlin.de> wrote in news:ep1srh$q5$1
@hahn.informatik.hu-berlin.de:

Hi,

a simple question: if I have a uniform random number generator (in
[0,1]) and I do a chi^2 goodness of fit test then I would reject the
Null hypothesis if the chi^2 value is larger than chi^2_{df;1-alpha}.
Usually I would never reject the Null hypothesis if the chi^2 value is
smaller than chi^2_{df;alpha/2} or larger than chi^2_{df;1-alpha/2}.
Could be any reason for such an approach?

(A non-statistician's answer:)
Appears to me that your current approach is too narrow. If you are
testing a RNG, then you should be testing at both alpha/2 and 1-alpha/2,
and maybe even other quantiles. The answer to whether to look at both
extremes depends on a proper formulation of what "should" happen. RNGs
should not be "too close" or "too far" from the theoretical result.

I am not convinced that this would be a general answer to the question of
whether to reject the null if you get a nearly predicted result and are
using a chi-square test, but I think it is fair when you are doing
simulations which have a well-characterized mathematical form.

There is the question "The Chi^2 of what?" as there are many possible
tests
of a PSEUDO random number generator. You hope that the tests you use will
allow you to have confidence that the generator will also be suitably
random for your use. Ain't always true!

In fact the good tests will repeat the sampling for the Chi^2s and ask
whether
there are a suitable number of failures on both sides. Some would go so
far as
to ask if the whole distriibution of the observed Chi^2s has the predicted
form.

A standard battery of tests is to have r, for resolution, bins in d
dimensions
for a total of r^d cells and then test for uniformity. You will notice
that
the cycle length required to support this gets very large very fast. In
the
special case where you can do the number theory and you use the full cycle
this is called the spectral test. (The name is mathematically justified
even
if it is not immediately the same as spectral analysis of time series.)

The tests in Knuth have to some extent have been viewed as being historic