### Navigation

- index
- modules |
- next |
- previous |
- Sage's Doctesting Framework »

# Classes involved in doctesting

This module controls the various classes involved in doctesting.

AUTHORS:

- David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.

class `sage.doctest.control.``DocTestController`(options, args)

  Bases: `sage.structure.sage_object.SageObject`

  This class controls doctesting of files.

  After creating it with appropriate options, call the `run()` method to run the doctests.

  `add_files`()

    Checks for the flags ‘–all’, ‘–new’ and ‘–sagenb’.

    For each one present, this function adds the appropriate directories and files to the todo list.

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: from sage.env import SAGE_SRC
        sage: import os
        sage: log_location = os.path.join(SAGE_TMP, 'control_dt_log.log')
        sage: DD = DocTestDefaults(all=True, logfile=log_location)
        sage: DC = DocTestController(DD, [])
        sage: DC.add_files()
        Doctesting entire Sage library.
        sage: os.path.join(SAGE_SRC, 'sage') in DC.files
        True

        sage: DD = DocTestDefaults(new = True)
        sage: DC = DocTestController(DD, [])
        sage: DC.add_files()
        Doctesting ...

        sage: DD = DocTestDefaults(sagenb = True)
        sage: DC = DocTestController(DD, [])
        sage: DC.add_files()
        Doctesting the Sage notebook.
        sage: DC.files[0][-6:]
        'sagenb'

  `cleanup`(final=True)

    Runs cleanup activities after actually running doctests.

    In particular, saves the stats to disk and closes the logfile.

    INPUT:

    - `final` – whether to close the logfile

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: from sage.env import SAGE_SRC
        sage: import os
        sage: dirname = os.path.join(SAGE_SRC, 'sage', 'rings', 'infinity.py')
        sage: DD = DocTestDefaults()

        sage: DC = DocTestController(DD, [dirname])
        sage: DC.expand_files_into_sources()
        sage: DC.sources.sort(key=lambda s:s.basename)

        sage: for i, source in enumerate(DC.sources):
        ....:     DC.stats[source.basename] = {'walltime': 0.1*(i+1)}
        ....:

        sage: DC.run()
        Running doctests with ID ...
        Doctesting 1 file.
        sage -t .../rings/infinity.py
            [... tests, ... s]
        ----------------------------------------------------------------------
        All tests passed!
        ----------------------------------------------------------------------
        Total time for all tests: ... seconds
            cpu time: ... seconds
            cumulative wall time: ... seconds
        0
        sage: DC.cleanup()

  `create_run_id`()

    Creates the run id.

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: DC = DocTestController(DocTestDefaults(), [])
        sage: DC.create_run_id()
        Running doctests with ID ...

  `expand_files_into_sources`()

    Expands `self.files`, which may include directories, into a list of `sage.doctest.FileDocTestSource`

    This function also handles the optional command line option.

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: from sage.env import SAGE_SRC
        sage: import os
        sage: dirname = os.path.join(SAGE_SRC, 'sage', 'doctest')
        sage: DD = DocTestDefaults(optional='all')
        sage: DC = DocTestController(DD, [dirname])
        sage: DC.expand_files_into_sources()
        sage: len(DC.sources)
        9
        sage: DC.sources[0].options.optional
        True

        sage: DD = DocTestDefaults(optional='magma,guava')
        sage: DC = DocTestController(DD, [dirname])
        sage: DC.expand_files_into_sources()
        sage: sorted(list(DC.sources[0].options.optional))
        ['guava', 'magma']

    We check that files are skipped appropriately:

        sage: dirname = tmp_dir()
        sage: filename = os.path.join(dirname, 'not_tested.py')
        sage: with open(filename, 'w') as F:
        ....:     F.write("#"*80 + "\n\n\n\n## nodoctest\n    sage: 1+1\n    4")
        sage: DC = DocTestController(DD, [dirname])
        sage: DC.expand_files_into_sources()
        sage: DC.sources
        []

    The directory `sage/doctest/tests` contains `nodoctest.py` but the files should still be tested when that directory is explicitly given (as opposed to being recursed into):

        sage: DC = DocTestController(DD, [os.path.join(SAGE_SRC, 'sage', 'doctest', 'tests')])
        sage: DC.expand_files_into_sources()
        sage: len(DC.sources) >= 10
        True

  `filter_sources`()

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: from sage.env import SAGE_SRC
        sage: import os
        sage: dirname = os.path.join(SAGE_SRC, 'sage', 'doctest')
        sage: DD = DocTestDefaults(failed=True)
        sage: DC = DocTestController(DD, [dirname])
        sage: DC.expand_files_into_sources()
        sage: for i, source in enumerate(DC.sources):
        ...       DC.stats[source.basename] = {'walltime': 0.1*(i+1)}
        sage: DC.stats['sage.doctest.control'] = {'failed':True,'walltime':1.0}
        sage: DC.filter_sources()
        Only doctesting files that failed last test.
        sage: len(DC.sources)
        1

  `load_stats`(filename)

    Load stats from the most recent run(s).

    Stats are stored as a JSON file, and include information on which files failed tests and the walltime used for execution of the doctests.

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: DC = DocTestController(DocTestDefaults(), [])
        sage: import json
        sage: filename = tmp_filename()
        sage: with open(filename, 'w') as stats_file:
        ...       json.dump({'sage.doctest.control':{u'walltime':1.0r}}, stats_file)
        sage: DC.load_stats(filename)
        sage: DC.stats['sage.doctest.control']
        {u'walltime': 1.0}

    If the file doesn’t exist, nothing happens. If there is an error, print a message. In any case, leave the stats alone:

        sage: d = tmp_dir()
        sage: DC.load_stats(os.path.join(d))  # Cannot read a directory
        Error loading stats from ...
        sage: DC.load_stats(os.path.join(d, "no_such_file"))
        sage: DC.stats['sage.doctest.control']
        {u'walltime': 1.0}

  `log`(s, end='n')

    Logs the string `s + end` (where `end` is a newline by default) to the logfile and prints it to the standard output.

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: DD = DocTestDefaults(logfile=tmp_filename())
        sage: DC = DocTestController(DD, [])
        sage: DC.log("hello world")
        hello world
        sage: DC.logfile.close()
        sage: print open(DD.logfile).read()
        hello world

    Check that no duplicate logs appear, even when forking (trac ticket #15244):

        sage: DD = DocTestDefaults(logfile=tmp_filename())
        sage: DC = DocTestController(DD, [])
        sage: DC.log("hello world")
        hello world
        sage: if os.fork() == 0:
        ....:     DC.logfile.close()
        ....:     os._exit(0)
        sage: DC.logfile.close()
        sage: print open(DD.logfile).read()
        hello world

  `run`()

    This function is called after initialization to set up and run all doctests.

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: from sage.env import SAGE_SRC
        sage: import os
        sage: DD = DocTestDefaults()
        sage: filename = os.path.join(SAGE_SRC, "sage", "sets", "non_negative_integers.py")
        sage: DC = DocTestController(DD, [filename])
        sage: DC.run()
        Running doctests with ID ...
        Doctesting 1 file.
        sage -t .../sage/sets/non_negative_integers.py
            [... tests, ... s]
        ----------------------------------------------------------------------
        All tests passed!
        ----------------------------------------------------------------------
        Total time for all tests: ... seconds
            cpu time: ... seconds
            cumulative wall time: ... seconds
        0

  `run_doctests`()

    Actually runs the doctests.

    This function is called by `run()`.

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: from sage.env import SAGE_SRC
        sage: import os
        sage: dirname = os.path.join(SAGE_SRC, 'sage', 'rings', 'homset.py')
        sage: DD = DocTestDefaults()
        sage: DC = DocTestController(DD, [dirname])
        sage: DC.expand_files_into_sources()
        sage: DC.run_doctests()
        Doctesting 1 file.
        sage -t .../sage/rings/homset.py
            [... tests, ... s]
        ----------------------------------------------------------------------
        All tests passed!
        ----------------------------------------------------------------------
        Total time for all tests: ... seconds
            cpu time: ... seconds
            cumulative wall time: ... seconds

  `run_val_gdb`(testing=False)

    Spawns a subprocess to run tests under the control of gdb or valgrind.

    INPUT:

    - `testing` – boolean; if True then the command to be run will be printed rather than a subprocess started.

    EXAMPLES:

    Note that the command lines include unexpanded environment variables. It is safer to let the shell expand them than to expand them here and risk insufficient quoting.

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: DD = DocTestDefaults(gdb=True)
        sage: DC = DocTestController(DD, ["hello_world.py"])
        sage: DC.run_val_gdb(testing=True)
        exec gdb -x "$SAGE_LOCAL/bin/sage-gdb-commands" --args python "$SAGE_LOCAL/bin/sage-runtests" --serial --timeout=0 hello_world.py

        sage: DD = DocTestDefaults(valgrind=True, optional="all", timeout=172800)
        sage: DC = DocTestController(DD, ["hello_world.py"])
        sage: DC.run_val_gdb(testing=True)
        exec valgrind --tool=memcheck --leak-resolution=high --leak-check=full --num-callers=25 --suppressions="$SAGE_LOCAL/lib/valgrind/sage.supp"  --log-file=".../valgrind/sage-memcheck.%p" python "$SAGE_LOCAL/bin/sage-runtests" --serial --timeout=172800 --optional=True hello_world.py

  `save_stats`(filename)

    Save stats from the most recent run as a JSON file.

    WARNING: This function overwrites the file.

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: DC = DocTestController(DocTestDefaults(), [])
        sage: DC.stats['sage.doctest.control'] = {u'walltime':1.0r}
        sage: filename = tmp_filename()
        sage: DC.save_stats(filename)
        sage: import json
        sage: D = json.load(open(filename))
        sage: D['sage.doctest.control']
        {u'walltime': 1.0}

  `second_on_modern_computer`()

    Return the wall time equivalent of a second on a modern computer.

    OUTPUT:

    Float. The wall time on your computer that would be equivalent to one second on a modern computer. Unless you have kick-ass hardware this should always be >= 1.0. Raises a `RuntimeError` if there are no stored timings to use as benchmark.

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: DC = DocTestController(DocTestDefaults(), [])
        sage: DC.second_on_modern_computer()   # not tested

  `sort_sources`()

    This function sorts the sources so that slower doctests are run first.

    EXAMPLES:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: from sage.env import SAGE_SRC
        sage: import os
        sage: dirname = os.path.join(SAGE_SRC, 'sage', 'doctest')
        sage: DD = DocTestDefaults(nthreads=2)
        sage: DC = DocTestController(DD, [dirname])
        sage: DC.expand_files_into_sources()
        sage: DC.sources.sort(key=lambda s:s.basename)
        sage: for i, source in enumerate(DC.sources):
        ...       DC.stats[source.basename] = {'walltime': 0.1*(i+1)}
        sage: DC.sort_sources()
        Sorting sources by runtime so that slower doctests are run first....
        sage: print "\n".join([source.basename for source in DC.sources])
        sage.doctest.util
        sage.doctest.test
        sage.doctest.sources
        sage.doctest.reporting
        sage.doctest.parsing
        sage.doctest.forker
        sage.doctest.control
        sage.doctest.all
        sage.doctest

  `test_safe_directory`(dir=None)

    Test that the given directory is safe to run Python code from.

    We use the check added to Python for this, which gives a warning when the current directory is considered unsafe. We promote this warning to an error with `-Werror`. See `sage/tests/cmdline.py` for a doctest that this works, see also trac ticket #13579.

    TESTS:

        sage: from sage.doctest.control import DocTestDefaults, DocTestController
        sage: DD = DocTestDefaults()
        sage: DC = DocTestController(DD, [])
        sage: DC.test_safe_directory()
        sage: d = os.path.join(tmp_dir(), "test")
        sage: os.mkdir(d)
        sage: os.chmod(d, 0o777)
        sage: DC.test_safe_directory(d)
        Traceback (most recent call last):
        ...
        RuntimeError: refusing to run doctests...

class `sage.doctest.control.``DocTestDefaults`(**kwds)

  Bases: `sage.structure.sage_object.SageObject`

  This class is used for doctesting the Sage doctest module.

  It fills in attributes to be the same as the defaults defined in `SAGE_LOCAL/bin/sage-runtests`, expect for a few places, which is mostly to make doctesting more predictable.

  EXAMPLES:

      sage: from sage.doctest.control import DocTestDefaults
      sage: D = DocTestDefaults()
      sage: D
      DocTestDefaults()
      sage: D.timeout
      -1

  Keyword arguments become attributes:

      sage: D = DocTestDefaults(timeout=100)
      sage: D
      DocTestDefaults(timeout=100)
      sage: D.timeout
      100

`sage.doctest.control.``run_doctests`(module, options=None)

  Runs the doctests in a given file.

  INPUTS:

  - `module` – a Sage module, a string, or a list of such.
  - `options` – a DocTestDefaults object or None.

  EXAMPLES:

      sage: run_doctests(sage.rings.infinity)
      Running doctests with ID ...
      Doctesting 1 file.
      sage -t .../sage/rings/infinity.py
          [... tests, ... s]
      ----------------------------------------------------------------------
      All tests passed!
      ----------------------------------------------------------------------
      Total time for all tests: ... seconds
          cpu time: ... seconds
          cumulative wall time: ... seconds

`sage.doctest.control.``skipdir`(dirname)

  Return True if and only if the directory `dirname` should not be doctested.

  EXAMPLES:

      sage: from sage.doctest.control import skipdir
      sage: skipdir(sage.env.SAGE_SRC)
      False
      sage: skipdir(os.path.join(sage.env.SAGE_SRC, "sage", "doctest", "tests"))
      True

`sage.doctest.control.``skipfile`(filename)

  Return True if and only if the file `filename` should not be doctested.

  EXAMPLES:

      sage: from sage.doctest.control import skipfile
      sage: skipfile("skipme.c")
      True
      sage: f = tmp_filename(ext=".pyx")
      sage: skipfile(f)
      False
      sage: open(f, "w").write("# nodoctest")
      sage: skipfile(f)
      True

### Navigation

- index
- modules |
- next |
- previous |
- Sage's Doctesting Framework »