Python: custom deepcopy in class hierarchy

Question

I have a class hierarchy representing linear algebra expression (as expression trees), something like this (it’s actually more complicated than that, but that should be enough to give you an idea).

• Expression
  • Operator
    • Times
    • Plus
  • Symbol
    • Matrix
    • Scalar

I’m manipulating those expression in many different ways, so I have to copy those expression a lot because I want to explore different ways of manipulating them (I really don’t see any way around copying them). Not surprisingly, copy.deepcopy() is slow.

Not all class attributes need do be deepcopied, and those expression trees will always be trees (if not, something is fundamentally wrong), so I can probably save some time by simplifying the memoization (or don’t use it at all).

I want to make copying faster by implementing a custom copy function. This could either mean implementing __deepcopy__(), writing an entirely new function or using something like get_state and set_state, I really don’t care. At the moment, I don’t care about pickling.

I’m using Python 3.

Since I have this class hierarchy where new attributes are introduced at different levels, I would prefer not to start from scratch on each level. Instead, it would be desirable to reuse some functionality of the superclass similar to how in __init__, one usually calls __init__ of the superclass. However, I would prefer not to call __init__ because that sometimes does some extra stuff that is not necessary when copying.

How do I do that in the fastest and most pythonic way? I was not able to find any reasonable guidelines regarding how to implement this copy functionality under those circumstances. I looked at the implementation of deepcopy both in Python 2 and 3. In Python 2, a bunch of different methods are used (get_state/set_state, using __dict__,…), in Python 3, I’m not able to find the corresponding function at all.

Answer 1

I found one possible solution myself:

class Expression(object):
    ...
    def __deepcopy__(self, memo):
        cpy = object.__new__(type(self))
        # manually copy all attributes of Expression here
        return cpy

class Operator(Expression):
    ...
    def __deepcopy__(self, memo):
        cpy = Expression.__deepcopy__(self, memo)
        # manually copy all attributes exclusive to Operator here
        return cpy

The new object is always created in __deepcopy__ of Expression. By using object.__new__(type(self)), the new object has the type of the object on which __deepcopy__ was originally called (for example Operator).

Unfortunately, this solution is not much faster than the default deepcopy. I'm wondering if this is because of all those calls to __deepcopy__ all the way to the top of the class hierarchy. I will investigate this.