### Navigation

- index
- modules |
- next |
- previous |
- Miscellaneous »

# Random testing

Some Sage modules do random testing in their doctests; that is, they construct test cases using a random number generator. To get the broadest possible test coverage, we want everybody who runs the doctests to use a different random seed; but we also want to be able to reproduce the problems when debugging. This module provides a decorator to help write random testers that meet these goals.

`sage.misc.random_testing.``random_testing`(fn)

  This decorator helps create random testers. These can be run as part of the standard Sage test suite; everybody who runs the test will use a different random number seed, so many different random tests will eventually be run.

  INPUT:

  > - `fn` - The function that we are wrapping for random testing.

  The resulting function will take two additional arguments, seed (default `None`) and print_seed (default `False`). The result will set the random number seed to the given seed value (or to a truly random value, if seed is not specified), then call the original function. If print_seed is true, then the seed will be printed before calling the original function. If the original function raises an exception, then the random seed that was used will be displayed, along with a message entreating the user to submit a bug report. All other arguments will be passed through to the original function.

  Here is a set of recommendations for using this wrapper.

  The function to be tested should take arguments specifying the difficulty of the test (size of the test cases, number of iterations, etc.), as well as an argument verbose (defaulting to false). With verbose true, it should print the values being tested. Suppose `test_foo()` takes an argument for number of iterations. Then the doctests could be:

      test_foo(2, verbose=True, seed=0)
      test_foo(10)
      test_foo(100) # long time

  The first doctest, with the specified seed and `verbose=True`, simply verifies that the tests really are reproducible (that `test_foo` is correctly using the `randstate` framework). The next two tests use truly random seeds, and will print out the seed used if the test fails (raises an exception).

  If you want a very long-running test using this setup, you should do something like:

      for _ in xrange(10^10): test_foo(100)

  instead of:

      test_foo(10^12)

  If the test fails after several hours, the latter snippet would make you rerun the test for several hours while reproducing and debugging the problem. With the former snippet, you only need to rerun `test_foo(100)` with a known-failing random seed.

  See `sage.misc.random_testing.test_add_commutes()` for a simple example using this decorator, and `sage.rings.tests` for realistic uses.

  Setting print_seed to true is useless in doctests, because the random seed printed will never match the expected doctest result (and using `# random` means the doctest framework will never report an error even if one happens). However, it is useful if you have a random test that sometimes segfaults. The normal print-the-random-seed-on-exceptions won’t work then, so you can run:

      while True: test_foo(print_seed=True)

  and look at the last seed that was printed before it crashed.

  TESTS:

      sage: from sage.misc.random_testing import random_testing
      sage: def foo(verbose=False):
      ...       'oh look, a docstring'
      ...       n = ZZ.random_element(2^50)
      ...       if verbose:
      ...           print "Random value: %s" % n
      ...       assert(n == 49681376900427)
      sage: foo = random_testing(foo)
      sage: foo(seed=0, verbose=True)
      Random value: 49681376900427
      sage: foo(seed=15, verbose=True)
      Random value: 1049538412064764
      Random testing has revealed a problem in foo
      Please report this bug!  You may be the first
      person in the world to have seen this problem.
      Please include this random seed in your bug report:
      Random seed: 15
      AssertionError()
      sage: foo() # random
      Random testing has revealed a problem in foo
      Please report this bug!  You may be the first
      person in the world to have seen this problem.
      Please include this random seed in your bug report:
      Random seed: 272500700755151445506092479579811710040
      AssertionError()
      sage: foo.__doc__
      'oh look, a docstring'
      sage: foo.__name__
      'foo'
      sage: def bar(): pass
      sage: bar = random_testing(bar)
      sage: bar(print_seed=True) # random
      Random seed: 262841091890156346923539765543814146051

`sage.misc.random_testing.``test_add_commutes`(*args, **kwargs)

  This is a simple demonstration of the `random_testing()` decorator and its recommended usage.

  We test that addition is commutative over rationals.

  EXAMPLES:

      sage: from sage.misc.random_testing import test_add_commutes
      sage: test_add_commutes(2, verbose=True, seed=0)
      a == -4, b == 0 ...
      Passes!
      a == -1/2, b == -1/95 ...
      Passes!
      sage: test_add_commutes(10)
      sage: test_add_commutes(1000) # long time

`sage.misc.random_testing.``test_add_is_mul`(*args, **kwargs)

  This example demonstrates a failing `random_testing()` test, and shows how to reproduce the error.

  DO NOT USE THIS AS AN EXAMPLE OF HOW TO USE `random_testing()`! Instead, look at `sage.misc.random_testing.test_add_commutes()`.

  We test that `a+b == a*b`, for a, b rational. This is of course false, so the test will almost always fail.

  EXAMPLES:

      sage: from sage.misc.random_testing import test_add_is_mul

  We start by testing that we get reproducible results when setting seed to 0.

      sage: test_add_is_mul(2, verbose=True, seed=0)
      a == -4, b == 0 ...
      Random testing has revealed a problem in test_add_is_mul
      Please report this bug!  You may be the first
      person in the world to have seen this problem.
      Please include this random seed in your bug report:
      Random seed: 0
      AssertionError()

  Normally in a `@random_testing` doctest, we would leave off the `verbose=True` and the `# random`. We put it in here so that we can verify that we are seeing the exact same error when we reproduce the error below.

      sage: test_add_is_mul(10, verbose=True) # random
      a == -2/7, b == 1 ...
      Random testing has revealed a problem in test_add_is_mul
      Please report this bug!  You may be the first
      person in the world to have seen this problem.
      Please include this random seed in your bug report:
      Random seed: 216390410596009428782506007128692114173
      AssertionError()

  OK, now assume that some user has reported a `test_add_is_mul()` failure. We can specify the same random_seed that was found in the bug report, and we will get the exact same failure so that we can debug the “problem”.

      sage: test_add_is_mul(10, verbose=True, seed=216390410596009428782506007128692114173)
      a == -2/7, b == 1 ...
      Random testing has revealed a problem in test_add_is_mul
      Please report this bug!  You may be the first
      person in the world to have seen this problem.
      Please include this random seed in your bug report:
      Random seed: 216390410596009428782506007128692114173
      AssertionError()

### Navigation

- index
- modules |
- next |
- previous |
- Miscellaneous »