Defining an inner class using the Python C-API

Question

In Python, it’s straightforward to define an inner class:

class MyClass(object):

    class MyInnerClass(object):
        pass

… which the inner class can be accessed as one would expect, e.g. by doing MyClass.MyInnerClass.

I am trying to set up something similar with an extension module. Typically one adds the extension types one defines to the extension module object in the modules’ <modulename>init() function with code like this:

/// …
if (PyType_Ready(&BufferModel_Type) < 0)      { return; }

/// Add the BufferModel type object to the module
Py_INCREF(&BufferModel_Type);
PyModule_AddObject(module,
    "Buffer",
    (PyObject*)&BufferModel_Type);

/// …

In order to set up the inner class, I varied this approach to try and add a PyTypeObject* as an attribute of another PyTypeObject*, like so:

/// …
if (PyType_Ready(&ImageBufferModel_Type) < 0) { return; }
if (PyType_Ready(&ImageModel_Type) < 0)       { return; }

/// Add the ImageBufferModel type object to im.Image
Py_INCREF(&ImageBufferModel_Type);
PyObject_SetAttrString((PyObject*)&ImageModel_Type,
    "ImageBuffer",
    (PyObject*)&ImageBufferModel_Type);
PyType_Modified((PyTypeObject*)&ImageModel_Type);

/// Add the ImageModel type object to the module
Py_INCREF(&ImageModel_Type);
PyModule_AddObject(module,
    "Image",
    (PyObject*)&ImageModel_Type);

/// …

… I figured PyObject_SetAttrString() would work as the introduction to “Type Objects” in the C-API docs specifically says:

Type objects can be handled using any of the PyObject_*() or PyType_*() functions […]

… and I added the call PyType_Modified() based on its description in the docs. But so: when I compile everything and try to load the extension, I get this error:

>>> import im
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    import im
  File "im/__init__.py", line 2, in <module>
    from im import (
TypeError: can't set attributes of built-in/extension type 'im.Image'

… I presume I am going about this the wrong way; what should I try instead?

tynn · Accepted Answer · 2016-03-23 00:40:02Z

For this you need to use tp_dict directly:

This field should normally be initialized to NULL before PyType_Ready is called; it may also be initialized to a dictionary containing initial attributes for the type. Once PyType_Ready() has initialized the type, extra attributes for the type may be added to this dictionary only if they don’t correspond to overloaded operations (like __add__()).

Instead of using PyObject_SetAttrString() you could just do

PyDict_SetItemString(ImageModel_Type.tp_dict, "ImageBuffer", (PyObject*) &ImageModel_Type);

But in this case the warning from the documentation applies:

It is not safe to use PyDict_SetItem() on or otherwise modify tp_dict with the dictionary C-API.

So maybe initialize the tp_dict before calling PyType_Ready on ImageModel_Type:

/// Initialize tp_dict with empty dictionary
ImageModel_Type.tp_dict = PyDict_New();
if (!ImageModel_Type.tp_dict) { return; }

/// Add the ImageBufferModel type object to im.Image
if (PyType_Ready(&ImageBufferModel_Type) < 0) { return; }
Py_INCREF(&ImageBufferModel_Type);
PyDict_SetItemString(ImageModel_Type.tp_dict,
    "ImageBuffer",
    (PyObject*)&ImageBufferModel_Type);

/// Add the ImageModel type object to the module
if (PyType_Ready(&ImageModel_Type) < 0) { return; }
Py_INCREF(&ImageModel_Type);
PyModule_AddObject(module,
    "Image",
    (PyObject*)&ImageModel_Type);

Wow, thanks! The type object documentation is sometimes confusing and I appreciate this direct answer, this totally works.
I should also add that making changes to a PyTypeObject’s tp_dict data member can be used to achieve many of the same ends often achieved with metaclasses, on the Python side (e.g. synthesizing class-based descriptors á la Django’s ORM, et cetera).

Collectives™ on Stack Overflow

Defining an inner class using the Python C-API

1 Answer 1

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Related