### Navigation

- index
- modules |
- next |
- previous |
- Sage's Doctesting Framework »

# Parsing docstrings

This module contains functions and classes that parse docstrings.

AUTHORS:

- David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
- Jeroen Demeyer(2014-08-28) – much improved handling of tolerances using interval arithmetic (trac ticket #16889).

class `sage.doctest.parsing.``MarkedOutput`

  Bases: `str`

  A subclass of string with context for whether another string matches it.

  EXAMPLES:

      sage: from sage.doctest.parsing import MarkedOutput
      sage: s = MarkedOutput("abc")
      sage: s.rel_tol
      0
      sage: s.update(rel_tol = .05)
      'abc'
      sage: s.rel_tol
      0.0500000000000000

  `update`(**kwds)

    EXAMPLES:

        sage: from sage.doctest.parsing import MarkedOutput
        sage: s = MarkedOutput("0.0007401")
        sage: s.update(abs_tol = .0000001)
        '0.0007401'
        sage: s.rel_tol
        0
        sage: s.abs_tol
        1.00000000000000e-7

class `sage.doctest.parsing.``OriginalSource`(example)

  Context swapping out the pre-parsed source with the original for better reporting.

  EXAMPLES:

      sage: from sage.doctest.sources import FileDocTestSource
      sage: from sage.doctest.control import DocTestDefaults
      sage: from sage.env import SAGE_SRC
      sage: import os
      sage: filename = os.path.join(SAGE_SRC,'sage','doctest','forker.py')
      sage: FDS = FileDocTestSource(filename,DocTestDefaults())
      sage: doctests, extras = FDS.create_doctests(globals())
      sage: ex = doctests[0].examples[0]
      sage: ex.sage_source
      'doctest_var = 42; doctest_var^2\n'
      sage: ex.source
      'doctest_var = Integer(42); doctest_var**Integer(2)\n'
      sage: from sage.doctest.parsing import OriginalSource
      sage: with OriginalSource(ex):
      ...       ex.source
      'doctest_var = 42; doctest_var^2\n'

class `sage.doctest.parsing.``SageDocTestParser`(long=False, optional_tags=())

  Bases: `doctest.DocTestParser`

  A version of the standard doctest parser which handles Sage’s custom options and tolerances in floating point arithmetic.

  `parse`(string, *args)

    A Sage specialization of `doctest.DocTestParser`.

    INPUTS:

    - `string` – the string to parse.
    - `name` – optional string giving the name indentifying string, to be used in error messages.

    OUTPUTS:

    - A list consisting of strings and `doctest.Example` instances. There will be at least one string between successive examples (exactly one unless or long or optional tests are removed), and it will begin and end with a string.

    EXAMPLES:

        sage: from sage.doctest.parsing import SageDocTestParser
        sage: DTP = SageDocTestParser(True, ('sage','magma','guava'))
        sage: example = 'Explanatory text::\n\n    sage: E = magma("EllipticCurve([1, 1, 1, -10, -10])") # optional: magma\n\nLater text'
        sage: parsed = DTP.parse(example)
        sage: parsed[0]
        'Explanatory text::\n\n'
        sage: parsed[1].sage_source
        'E = magma("EllipticCurve([1, 1, 1, -10, -10])") # optional: magma\n'
        sage: parsed[2]
        '\nLater text'

    If the doctest parser is not created to accept a given optional argument, the corresponding examples will just be removed:

        sage: DTP2 = SageDocTestParser(True, ('sage',))
        sage: parsed2 = DTP2.parse(example)
        sage: parsed2
        ['Explanatory text::\n\n', '\nLater text']

    You can mark doctests as having a particular tolerance:

        sage: example2 = 'sage: gamma(1.6) # tol 2.0e-11\n0.893515349287690'
        sage: ex = DTP.parse(example2)[1]
        sage: ex.sage_source
        'gamma(1.6) # tol 2.0e-11\n'
        sage: ex.want
        '0.893515349287690\n'
        sage: type(ex.want)
        <class 'sage.doctest.parsing.MarkedOutput'>
        sage: ex.want.tol
        2.000000000000000000?e-11

    You can use continuation lines:

        sage: s = "sage: for i in range(4):\n....:     print i\n....:\n"
        sage: ex = DTP2.parse(s)[1]
        sage: ex.source
        'for i in range(Integer(4)):\n    print i\n'

    Sage currently accepts backslashes as indicating that the end of the current line should be joined to the next line. This feature allows for breaking large integers over multiple lines but is not standard for Python doctesting. It’s not guaranteed to persist, but works in Sage 5.5:

        sage: n = 1234\
        ....:     5678
        sage: print n
        12345678
        sage: type(n)
        <type 'sage.rings.integer.Integer'>

    It also works without the line continuation:

        sage: m = 8765\
        4321
        sage: print m
        87654321

class `sage.doctest.parsing.``SageOutputChecker`

  Bases: `doctest.OutputChecker`

  A modification of the doctest OutputChecker that can check relative and absolute tolerance of answers.

  EXAMPLES:

      sage: from sage.doctest.parsing import SageOutputChecker, MarkedOutput, SageDocTestParser
      sage: import doctest
      sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
      sage: DTP = SageDocTestParser(True, ('sage','magma','guava'))
      sage: OC = SageOutputChecker()
      sage: example2 = 'sage: gamma(1.6) # tol 2.0e-11\n0.893515349287690'
      sage: ex = DTP.parse(example2)[1]
      sage: ex.sage_source
      'gamma(1.6) # tol 2.0e-11\n'
      sage: ex.want
      '0.893515349287690\n'
      sage: type(ex.want)
      <class 'sage.doctest.parsing.MarkedOutput'>
      sage: ex.want.tol
      2.000000000000000000?e-11
      sage: OC.check_output(ex.want, '0.893515349287690', optflag)
      True
      sage: OC.check_output(ex.want, '0.8935153492877', optflag)
      True
      sage: OC.check_output(ex.want, '0', optflag)
      False
      sage: OC.check_output(ex.want, 'x + 0.8935153492877', optflag)
      False

  `add_tolerance`(wantval, want)

    Enlarge the real interval element `wantval` according to the tolerance options in `want`.

    INPUT:

    - `wantval` – a real interval element
    - `want` – a `MarkedOutput` describing the tolerance

    OUTPUT:

    - an interval element containing `wantval`

    EXAMPLES:

        sage: from sage.doctest.parsing import MarkedOutput, SageOutputChecker
        sage: OC = SageOutputChecker()
        sage: want_tol = MarkedOutput().update(tol=0.0001)
        sage: want_abs = MarkedOutput().update(abs_tol=0.0001)
        sage: want_rel = MarkedOutput().update(rel_tol=0.0001)
        sage: OC.add_tolerance(pi.n(64), want_tol).endpoints()
        (3.14127849432443, 3.14190681285516)
        sage: OC.add_tolerance(pi.n(64), want_abs).endpoints()
        (3.14149265358979, 3.14169265358980)
        sage: OC.add_tolerance(pi.n(64), want_rel).endpoints()
        (3.14127849432443, 3.14190681285516)
        sage: OC.add_tolerance(1e1000, want_tol)
        1.000?e1000
        sage: OC.add_tolerance(1e1000, want_abs)
        1.000000000000000?e1000
        sage: OC.add_tolerance(1e1000, want_rel)
        1.000?e1000
        sage: OC.add_tolerance(0, want_tol)
        0.000?
        sage: OC.add_tolerance(0, want_abs)
        0.000?
        sage: OC.add_tolerance(0, want_rel)
        0

  `check_output`(want, got, optionflags)

    Checks to see if the output matches the desired output.

    If `want` is a `MarkedOutput` instance, takes into account the desired tolerance.

    INPUT:

    - `want` – a string or `MarkedOutput`
    - `got` – a string
    - `optionflags` – an integer, passed down to `doctest.OutputChecker`

    OUTPUT:

    - boolean, whether `got` matches `want` up to the specified tolerance.

    EXAMPLES:

        sage: from sage.doctest.parsing import MarkedOutput, SageOutputChecker
        sage: import doctest
        sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
        sage: rndstr = MarkedOutput("I'm wrong!").update(random=True)
        sage: tentol = MarkedOutput("10.0").update(tol=.1)
        sage: tenabs = MarkedOutput("10.0").update(abs_tol=.1)
        sage: tenrel = MarkedOutput("10.0").update(rel_tol=.1)
        sage: zerotol = MarkedOutput("0.0").update(tol=.1)
        sage: zeroabs = MarkedOutput("0.0").update(abs_tol=.1)
        sage: zerorel = MarkedOutput("0.0").update(rel_tol=.1)
        sage: zero = "0.0"
        sage: nf = "9.5"
        sage: ten = "10.05"
        sage: eps = "-0.05"
        sage: OC = SageOutputChecker()

        sage: OC.check_output(rndstr,nf,optflag)
        True

        sage: OC.check_output(tentol,nf,optflag)
        True
        sage: OC.check_output(tentol,ten,optflag)
        True
        sage: OC.check_output(tentol,zero,optflag)
        False

        sage: OC.check_output(tenabs,nf,optflag)
        False
        sage: OC.check_output(tenabs,ten,optflag)
        True
        sage: OC.check_output(tenabs,zero,optflag)
        False

        sage: OC.check_output(tenrel,nf,optflag)
        True
        sage: OC.check_output(tenrel,ten,optflag)
        True
        sage: OC.check_output(tenrel,zero,optflag)
        False

        sage: OC.check_output(zerotol,zero,optflag)
        True
        sage: OC.check_output(zerotol,eps,optflag)
        True
        sage: OC.check_output(zerotol,ten,optflag)
        False

        sage: OC.check_output(zeroabs,zero,optflag)
        True
        sage: OC.check_output(zeroabs,eps,optflag)
        True
        sage: OC.check_output(zeroabs,ten,optflag)
        False

        sage: OC.check_output(zerorel,zero,optflag)
        True
        sage: OC.check_output(zerorel,eps,optflag)
        False
        sage: OC.check_output(zerorel,ten,optflag)
        False

    More explicit tolerance checks:

        sage: _ = x  # rel tol 1e10
        sage: raise RuntimeError   # rel tol 1e10
        Traceback (most recent call last):
        ...
        RuntimeError
        sage: 1  # abs tol 2
        -0.5
        sage: print "0.9999"    # rel tol 1e-4
        1.0
        sage: print "1.00001"   # abs tol 1e-5
        1.0
        sage: 0  # rel tol 1
        1

    Spaces before numbers or between the sign and number are ignored:

        sage: print "[ - 1, 2]"  # abs tol 1e-10
        [-1,2]

  `human_readable_escape_sequences`(string)

    Make ANSI escape sequences human readable.

    EXAMPLES:

        sage: print 'This is \x1b[1mbold\x1b[0m text'
        This is <CSI-1m>bold<CSI-0m> text

    TESTS:

        sage: from sage.doctest.parsing import SageOutputChecker
        sage: OC = SageOutputChecker()
        sage: teststr = '-'.join([
        ....:     'bold\x1b[1m',
        ....:     'red\x1b[31m',
        ....:     'oscmd\x1ba'])
        sage: OC.human_readable_escape_sequences(teststr)
        'bold<CSI-1m>-red<CSI-31m>-oscmd<ESC-a>'

  `output_difference`(example, got, optionflags)

    Report on the differences between the desired result and what was actually obtained.

    If `want` is a `MarkedOutput` instance, takes into account the desired tolerance.

    INPUT:

    - `example` – a `doctest.Example` instance
    - `got` – a string
    - `optionflags` – an integer, passed down to `doctest.OutputChecker`

    OUTPUT:

    - a string, describing how `got` fails to match `example.want`

    EXAMPLES:

        sage: from sage.doctest.parsing import MarkedOutput, SageOutputChecker
        sage: import doctest
        sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
        sage: tentol = doctest.Example('',MarkedOutput("10.0\n").update(tol=.1))
        sage: tenabs = doctest.Example('',MarkedOutput("10.0\n").update(abs_tol=.1))
        sage: tenrel = doctest.Example('',MarkedOutput("10.0\n").update(rel_tol=.1))
        sage: zerotol = doctest.Example('',MarkedOutput("0.0\n").update(tol=.1))
        sage: zeroabs = doctest.Example('',MarkedOutput("0.0\n").update(abs_tol=.1))
        sage: zerorel = doctest.Example('',MarkedOutput("0.0\n").update(rel_tol=.1))
        sage: tlist = doctest.Example('',MarkedOutput("[10.0, 10.0, 10.0, 10.0, 10.0, 10.0]\n").update(abs_tol=1.0))
        sage: zero = "0.0"
        sage: nf = "9.5"
        sage: ten = "10.05"
        sage: eps = "-0.05"
        sage: L = "[9.9, 8.7, 10.3, 11.2, 10.8, 10.0]"
        sage: OC = SageOutputChecker()

        sage: print OC.output_difference(tenabs,nf,optflag)
        Expected:
            10.0
        Got:
            9.5
        Tolerance exceeded:
            10.0 vs 9.5, tolerance 5e-01 > 1e-01

        sage: print OC.output_difference(tentol,zero,optflag)
        Expected:
            10.0
        Got:
            0.0
        Tolerance exceeded:
            10.0 vs 0.0, tolerance 1e+00 > 1e-01

        sage: print OC.output_difference(tentol,eps,optflag)
        Expected:
            10.0
        Got:
            -0.05
        Tolerance exceeded:
            10.0 vs -0.05, tolerance 1e+00 > 1e-01

        sage: print OC.output_difference(tlist,L,optflag)
        Expected:
            [10.0, 10.0, 10.0, 10.0, 10.0, 10.0]
        Got:
            [9.9, 8.7, 10.3, 11.2, 10.8, 10.0]
        Tolerance exceeded in 2 of 6:
            10.0 vs 8.7, tolerance 1e+00 > 1e+00
            10.0 vs 11.2, tolerance 1e+00 > 1e+00

    TESTS:

        sage: print OC.output_difference(tenabs,zero,optflag)
        Expected:
            10.0
        Got:
            0.0
        Tolerance exceeded:
            10.0 vs 0.0, tolerance 1e+01 > 1e-01

        sage: print OC.output_difference(tenrel,zero,optflag)
        Expected:
            10.0
        Got:
            0.0
        Tolerance exceeded:
            10.0 vs 0.0, tolerance 1e+00 > 1e-01

        sage: print OC.output_difference(tenrel,eps,optflag)
        Expected:
            10.0
        Got:
            -0.05
        Tolerance exceeded:
            10.0 vs -0.05, tolerance 1e+00 > 1e-01

        sage: print OC.output_difference(zerotol,ten,optflag)
        Expected:
            0.0
        Got:
            10.05
        Tolerance exceeded:
            0.0 vs 10.05, tolerance 1e+01 > 1e-01

        sage: print OC.output_difference(zeroabs,ten,optflag)
        Expected:
            0.0
        Got:
            10.05
        Tolerance exceeded:
            0.0 vs 10.05, tolerance 1e+01 > 1e-01

        sage: print OC.output_difference(zerorel,eps,optflag)
        Expected:
            0.0
        Got:
            -0.05
        Tolerance exceeded:
            0.0 vs -0.05, tolerance inf > 1e-01

        sage: print OC.output_difference(zerorel,ten,optflag)
        Expected:
            0.0
        Got:
            10.05
        Tolerance exceeded:
            0.0 vs 10.05, tolerance inf > 1e-01

`sage.doctest.parsing.``get_source`(example)

  Returns the source with the leading ‘sage: ‘ stripped off.

  EXAMPLES:

      sage: from sage.doctest.parsing import get_source
      sage: from sage.doctest.sources import DictAsObject
      sage: example = DictAsObject({})
      sage: example.sage_source = "2 + 2"
      sage: example.source = "sage: 2 + 2"
      sage: get_source(example)
      '2 + 2'
      sage: example = DictAsObject({})
      sage: example.source = "3 + 3"
      sage: get_source(example)
      '3 + 3'

`sage.doctest.parsing.``make_marked_output`(s, D)

  Auxilliary function for pickling.

  EXAMPLES:

      sage: from sage.doctest.parsing import make_marked_output
      sage: s = make_marked_output("0.0007401",{'abs_tol':.0000001})
      sage: s
      '0.0007401'
      sage: s.abs_tol
      1.00000000000000e-7

`sage.doctest.parsing.``parse_optional_tags`(string)

  Returns a set consisting of the optional tags from the following set that occur in a comment on the first line of the input string.

  - ‘long time’
  - ‘not implemented’
  - ‘not tested’
  - ‘known bug’
  - ‘optional: PKG_NAME’ – the set will just contain ‘PKG_NAME’

  EXAMPLES:

      sage: from sage.doctest.parsing import parse_optional_tags
      sage: parse_optional_tags("sage: magma('2 + 2')# optional: magma")
      {'magma'}
      sage: parse_optional_tags("sage: #optional -- mypkg")
      {'mypkg'}
      sage: parse_optional_tags("sage: print(1)  # parentheses are optional here")
      set()
      sage: parse_optional_tags("sage: print(1)  # optional")
      {''}
      sage: sorted(list(parse_optional_tags("sage: #optional -- foo bar, baz")))
      ['bar', 'foo']
      sage: sorted(list(parse_optional_tags("    sage: factor(10^(10^10) + 1) # LoNg TiME, NoT TeSTED; OptioNAL -- P4cka9e")))
      ['long time', 'not tested', 'p4cka9e']
      sage: parse_optional_tags("    sage: raise RuntimeError # known bug")
      {'bug'}
      sage: sorted(list(parse_optional_tags("    sage: determine_meaning_of_life() # long time, not implemented")))
      ['long time', 'not implemented']

  We don’t parse inside strings:

      sage: parse_optional_tags("    sage: print '  # long time'")
      set()
      sage: parse_optional_tags("    sage: print '  # long time'  # not tested")
      {'not tested'}

  UTF-8 works:

      sage: parse_optional_tags("'ěščřžýáíéďĎ'")
      set()

`sage.doctest.parsing.``parse_tolerance`(source, want)

  Returns a version of `want` marked up with the tolerance tags specified in `source`.

  INPUT:

  - `source` – a string, the source of a doctest
  - `want` – a string, the desired output of the doctest

  OUTPUT:

  - `want` if there are no tolerance tags specified; a `MarkedOutput` version otherwise.

  EXAMPLES:

      sage: from sage.doctest.parsing import parse_tolerance
      sage: marked = parse_tolerance("sage: s.update(abs_tol = .0000001)", "")
      sage: type(marked)
      <type 'str'>
      sage: marked = parse_tolerance("sage: s.update(tol = 0.1); s.rel_tol # abs tol     0.01 ", "")
      sage: marked.tol
      0
      sage: marked.rel_tol
      0
      sage: marked.abs_tol
      0.010000000000000000000?

`sage.doctest.parsing.``pre_hash`(s)

  Prepends a string with its length.

  EXAMPLES:

      sage: from sage.doctest.parsing import pre_hash
      sage: pre_hash("abc")
      '3:abc'

`sage.doctest.parsing.``reduce_hex`(fingerprints)

  Returns a symmetric function of the arguments as hex strings.

  The arguments should be 32 character strings consiting of hex digits: 0-9 and a-f.

  EXAMPLES:

      sage: from sage.doctest.parsing import reduce_hex
      sage: reduce_hex(["abc", "12399aedf"])
      '0000000000000000000000012399a463'
      sage: reduce_hex(["12399aedf","abc"])
      '0000000000000000000000012399a463'

### Navigation

- index
- modules |
- next |
- previous |
- Sage's Doctesting Framework »