1

I am having problems when JSON is trying to decode my serialized custom class with the class information encoded in the dict. I will try to provide as much information as possible, so please bear with the long post.

I am using the excellent tutorial provided here as reference. I am referring to the section which details writing encoding/decoding for your own classes.

My package-module structure is partially as follows:

+--- PyDev Project
    +-- src
        +-- astar
        |    +-- __init__.py
        |    +-- astar.py
        +-- common
        |    +-- __init__.py
        |    +-- service.py
        |    +-- graph.py
        |    +-- vertex.py

Each of these modules has its own class(es). I am serializing an object of the Service class in the common.service module. This is basically to use over a Pyro connection.

The serialization code is:

def convert_to_builtin_type(obj):
    print('default(', repr(obj), ')')
    # Convert objects to a dictionary of their representation
    d = { '__class__':obj.__class__.__name__, 
          '__module__':obj.__module__,
        }
d.update(obj.__dict__)
return d

The de-serialization code is:

def dict_to_object(self, d):
    if '__class__' in d:
        class_name = d.pop('__class__')
        module_name = d.pop('__module__')
        # module = __import__(module_name)
        module = import_module(module_name)
        print('MODULE:', module)
        class_ = getattr(module, class_name)
        print('CLASS:', class_)
        args = dict( (key.encode('ascii'), value) for key, value in d.items())
        print('INSTANCE ARGS:', args)
        inst = class_(**args)
    else:
        inst = d
    return inst

The problem is occurring during de-serialization. The local variable values are:

class_ = <module 'common.Service' from path-to-file.py>
class_name = 'Service'
module = <module 'common' from path-to-__init__.py>
module_name = 'common.Service'

The reason this is failing is because class_ is returning the package.module value and not the class name inside that module. Technically, class_ should hold common.service.Service which is the actual class. Due to this, the last statement inst = class(**args)_ is failing with a "TypeError: module object not callable" error.

I know that import or importlib.import_module both import that top level module, but in my case, how do I get to a class inside the second level module? Technically, the second level is the module and the first level is the package, so I need a class in the pkg.module that I am unable to get at.

I hope the question made sense and has enough research shown. I have more information if it needs to be clarified.

EDIT: Problem solved with User suggestion.

def dict_to_object(self, d):
    if '__class__' in d:
        class_name = d.pop('__class__')
        module_name = d.pop('__module__')
        # module = __import__(module_name)
        __import__(module_name)
        module = sys.modules[module_name]
        print('MODULE:', module)
        class_ = getattr(module, class_name)
        print('CLASS:', class_)
        args = dict((key, value) for key, value in d.items())
        print('INSTANCE ARGS:', args)
        inst = class_(**args)
    else:
        inst = d
    return inst

1 Answer 1

1

Ideas:

  1. It could be that you problem is in this line:

    import_module(module_name)
    

    reconsider the arguments you pass to that function.

  2. knowing the pickle serializer it does the following instead of import_module

    import sys
    __import__(module_name)
    module = sys.modules[module_name]
    
  3. In Python 3 try using also obj.__class__.__qualname__. it includes the module name and the nesting classes.

    class X:
         class Y: pass # __qualname__ includes X.Y
    
  4. I would be nice if you post the working code afterwards because this may be a problem for many people.

Sign up to request clarification or add additional context in comments.

1 Comment

Thank you for the inputs. The second method worked. Using sys.import helps import all the underlying modules (as for now). I am posting the working modified code in the original post.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.