### Navigation

- index
- modules |
- next |
- previous |
- Miscellaneous »

# Fixing Pickle for Nested Classes

Fixing Pickle for Nested Classes

As of Python 2.6, names for nested classes are set by Python in a way which is incompatible with the pickling of such classes (pickling by name):

    sage: class A:
    ...       class B:
    ...            pass
    sage: A.B.__name__
    'B'

instead of the a priori more natural `"A.B"`.

Furthermore, upon pickling (here in save_global) and unpickling (in load_global) a class with name `"A.B"` in a module `mod`, the standard cPickle module searches for `"A.B"` in `mod.__dict__` instead of looking up `"A"` and then `"B"` in the result.

See: http://groups.google.com/group/sage-devel/browse_thread/thread/6c7055f4a580b7ae/

This module provides two utilities to workaround this issue:

> - `nested_pickle()` “fixes” recursively the name of the subclasses of a class and inserts their fullname `"A.B"` in `mod.__dict__`
> - `NestedClassMetaclass` is a metaclass ensuring that `nested_pickle()` is called on a class upon creation.

See also `sage.misc.nested_class_test`.

EXAMPLES:

    sage: from sage.misc.nested_class import A1, nested_pickle

    sage: A1.A2.A3.__name__
    'A3'
    sage: A1.A2.A3
    <class sage.misc.nested_class.A3 at ...>

    sage: nested_pickle(A1)
    <class sage.misc.nested_class.A1 at ...>

    sage: A1.A2
    <class sage.misc.nested_class.A1.A2 at ...>

    sage: A1.A2.A3
    <class sage.misc.nested_class.A1.A2.A3 at ...>
    sage: A1.A2.A3.__name__
    'A1.A2.A3'

    sage: sage.misc.nested_class.__dict__['A1.A2'] is A1.A2
    True
    sage: sage.misc.nested_class.__dict__['A1.A2.A3'] is A1.A2.A3
    True

All of this is not perfect. In the following scenario:

    sage: class A1:
    ...       class A2:
    ...           pass
    sage: class B1:
    ...       A2 = A1.A2
    ...

The name for `"A1.A2"` could potentially be set to `"B1.A2"`. But that will work anyway.

`sage.misc.nested_class.``modify_for_nested_pickle`(cls, name_prefix, module, first_run=True)

  Modify the subclasses of the given class to be picklable, by giving them a mangled name and putting the mangled name in the module namespace.

  INPUTS:

  - `cls` - The class to modify.
  - `name_prefix` - The prefix to prepend to the class name.
  - `module` - The module object to modify with the mangled name.
  - `first_run` - optional bool (default True): Whether or not this function is run for the first time on `cls`.

  NOTE:

  This function would usually not be directly called. It is internally used in `NestedClassMetaclass`.

  EXAMPLES:

      sage: from sage.misc.nested_class import *
      sage: class A(object):
      ...       class B(object):
      ...           pass
      ...
      sage: module = sys.modules['__main__']
      sage: A.B.__name__
      'B'
      sage: getattr(module, 'A.B', 'Not found')
      'Not found'
      sage: modify_for_nested_pickle(A, 'A', sys.modules['__main__'])
      sage: A.B.__name__
      'A.B'
      sage: getattr(module, 'A.B', 'Not found')
      <class '__main__.A.B'>

  Here we demonstrate the effect of the `first_run` argument:

      sage: modify_for_nested_pickle(A, 'X', sys.modules['__main__'])
      sage: A.B.__name__ # nothing changed
      'A.B'
      sage: modify_for_nested_pickle(A, 'X', sys.modules['__main__'], first_run=False)
      sage: A.B.__name__
      'X.A.B'

  Note that the class is now found in the module under both its old and its new name:

      sage: getattr(module, 'A.B', 'Not found')
      <class '__main__.X.A.B'>
      sage: getattr(module, 'X.A.B', 'Not found')
      <class '__main__.X.A.B'>

  TESTS:

  The following is a real life example, that was enabled by the internal use of the``first_run`` in trac ticket #9107:

      sage: cython_code = [
      ....:  "from sage.structure.unique_representation import UniqueRepresentation",
      ....:  "class A1(UniqueRepresentation):",
      ....:  "    class B1(UniqueRepresentation):",
      ....:  "        class C1: pass",
      ....:  "    class B2:",
      ....:  "        class C2: pass"]
      sage: import os
      sage: cython(os.linesep.join(cython_code))

  Before trac ticket #9107, the name of `A1.B1.C1` would have been wrong:

      sage: A1.B1.C1.__name__
      'A1.B1.C1'
      sage: A1.B2.C2.__name__
      'A1.B2.C2'
      sage: A_module = sys.modules[A1.__module__]
      sage: getattr(A_module, 'A1.B1.C1', 'Not found').__name__
      'A1.B1.C1'
      sage: getattr(A_module, 'A1.B2.C2', 'Not found').__name__
      'A1.B2.C2'

`sage.misc.nested_class.``nested_pickle`(cls)

  This decorator takes a class that potentially contains nested classes. For each such nested class, its name is modified to a new illegal identifier, and that name is set in the module. For example, if you have:

      sage: from sage.misc.nested_class import nested_pickle
      sage: module = sys.modules['__main__']
      sage: class A(object):
      ...       class B:
      ...           pass
      sage: nested_pickle(A)
      <class '__main__.A'>

  then the name of class `"B"` will be modified to `"A.B"`, and the `"A.B"` attribute of the module will be set to class `"B"`:

      sage: A.B.__name__
      'A.B'
      sage: getattr(module, 'A.B', 'Not found')
      <class __main__.A.B at ...>

  In Python 2.6, decorators work with classes; then `@nested_pickle` should work as a decorator:

      sage: @nested_pickle    # todo: not implemented
      ...   class A2(object):
      ...       class B:
      ...           pass
      sage: A2.B.__name__    # todo: not implemented
      'A2.B'
      sage: getattr(module, 'A2.B', 'Not found')    # todo: not implemented
      <class __main__.A2.B at ...>

  EXAMPLES:

      sage: from sage.misc.nested_class import *
      sage: loads(dumps(MainClass.NestedClass())) # indirect doctest
      <sage.misc.nested_class.MainClass.NestedClass object at 0x...>

class `sage.misc.nested_class.``NestedClassMetaclass`

  Bases: `type`

  A metaclass for nested pickling.

  Check that one can use a metaclass to ensure nested_pickle is called on any derived subclass:

      sage: from sage.misc.nested_class import NestedClassMetaclass
      sage: class ASuperClass(object):
      ...       __metaclass__ = NestedClassMetaclass
      ...
      sage: class A3(ASuperClass):
      ...       class B(object):
      ...           pass
      ...
      sage: A3.B.__name__
      'A3.B'
      sage: getattr(sys.modules['__main__'], 'A3.B', 'Not found')
      <class '__main__.A3.B'>

class `sage.misc.nested_class.``MainClass`

  Bases: `object`

  A simple class to test nested_pickle.

  EXAMPLES:

      sage: from sage.misc.nested_class import *
      sage: loads(dumps(MainClass()))
      <sage.misc.nested_class.MainClass object at 0x...>

  class `NestedClass`

    Bases: `object`

    EXAMPLES:

        sage: from sage.misc.nested_class import *
        sage: loads(dumps(MainClass.NestedClass()))
        <sage.misc.nested_class.MainClass.NestedClass object at 0x...>

    class `NestedSubClass`

      Bases: `object`

      EXAMPLES:

          sage: from sage.misc.nested_class import *
          sage: loads(dumps(MainClass.NestedClass.NestedSubClass()))
          <sage.misc.nested_class.MainClass.NestedClass.NestedSubClass object at 0x...>
          sage: getattr(sage.misc.nested_class, 'MainClass.NestedClass.NestedSubClass')
          <class 'sage.misc.nested_class.MainClass.NestedClass.NestedSubClass'>
          sage: MainClass.NestedClass.NestedSubClass.__name__
          'MainClass.NestedClass.NestedSubClass'

### Navigation

- index
- modules |
- next |
- previous |
- Miscellaneous »