Why metaclass should inherit from type?

Question

We can use functions as metaclass and my understanding is they are not derived from type as shown below :

def test_meta(name, bases, atts):
    print("testmeta called for " + name)
    return type(name,bases,atts);

class Computer:
    __metaclass__ = test_meta
    def __init__(self,brand,model,price):
        self.brand = brand
        self.model = model
        self.price = price

#
#
ob = Computer('p1','core2duo',21985.25)

However when we write metaclass it should be inherited from type and i could not understand the reasoning behind this :

class MyMeta:
    def __new__(meta, name, bases, dct):
        print ('-----------------------------------')
        print ("Allocating memory for class", name)
        print (meta)
        print (bases)
        print (dct)
        return type.__new__(meta, name, bases, dct)

    def __init__(cls, name, bases, dct):
        print ('-----------------------------------')
        print ("Initializing class", name)
        print (cls)
        print (bases)
        print (dct)
        type.__init__(cls,name, bases, dct)

    def __call__(mcs, *args, **kwargs):
        print ('-----------------------------------')
        print ("calling class")
        print mcs

class Computer:
    __metaclass__ = MyMeta
    def __init__(self,brand,model,price):
        self.brand = brand
        self.model = model
        self.price = price

#
#
ob = Computer('p1','core2duo',21985.25)

For example in above code I could not understand why MyMeta class should be inherited from type when we are explicitly calling type functions i.e. new,init,call. Also the error "descriptor 'init' requires a 'type' object but received a 'instance'" is coming only when i=instance (ob) is created.

Answer 1

In ordinary Python code, the only call that actually creates an in memory class object, with the binary structure and all the fields needed is type.__new__. Using a function as callable as the metaclass will have to just instantiate type itself, or create something that is not a class at all (see bellow).

It would be possible to create another "base metaclass" using native C code, or even in pure Python, calling the native OS memory allocation functions and filling in the same structure defined for a "type object", - however, that would only accomplish exactly the same task type.__new__ already does, so it would just be a complicated and error prone reinventing of a wheel that can't be further improved, since the resulting binary layout must be the same as defined in that structure (i.e. pointers for the methods implementing the "magic fuctions", like __init__, __geitem__ and so on, have to be at the same offsets as in that structure)

(Additional fields, and even exquisite values for data in this structure could be accomplished by code inheriting type - therefore by an ordinary metaclass anyway)

As you put it, any callable - even a simple function, can be indicated as the "metaclass" on the class body (both in Python 2 and Python 3) - however, whatever this callable does, at a certain point it will have to call type.__new__ (functions do that indirectly by calling type). However, once the class is built, it is a Python object itself, which is an instance of a class - the class's type, (what is in it's __class__ attribute) is it effective metaclass. If the metaclass pointed in code as the __metaclass__ attribute or as the metaclass kwarg in Python 3, is a subclass of type, that will as well be the metaclass proper. Otherwise, if it is an ordinary function that just acts as a class factory calling type in its body, the effective metaclass is just type.

In other words:

In [1]: def function_meta(name, bases, ns):
   ...:     return type(name, bases, ns)
   ...: 

In [2]: class test(metaclass=function_meta):
   ...:     pass
   ...: 

In [3]: type(test)
Out[3]: type

In [4]: class ClassMeta(type):
   ...:     pass
   ...: 
   ...: 

In [5]: class test2(metaclass=ClassMeta):
   ...:     pass
   ...:

In [6]: type(test2)
Out[6]: __main__.ClassMeta

And so, why can't a class that don't inherit from type be used?

The problem is not the specified metaclass not inheriting from type - any callable can be used, as shown above. The error you get when you attempt to do that is due to calling type.__new__ with a non-subclass of type as the first parameter:

In [11]: class InheritFromOther(object):
    ...:     def __new__(mcls, name, bases, ns):
    ...:         return type.__new__(mcls, name, bases, ns)
    ...:         # the line above is the one that errors - as
    ...:         # mcls here is not a subclass of type.
    ...:         # type(name, bases, ns) would work.

In [12]: class test4(metaclass=InheritFromOther):
    ...:     pass
    ...: 
    ...: 
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-12-bf5b1fa4bb7f> in <module>()
----> 1 class test4(metaclass=InheritFromOther):
      2     pass

<ipython-input-11-62d1fe46490b> in __new__(mcls, name, bases, ns)
      1 class InheritFromOther(object):
      2     def __new__(mcls, name, bases, ns):
----> 3         return type.__new__(mcls, name, bases, ns)
      4 

TypeError: type.__new__(InheritFromOther): InheritFromOther is not a subtype of type

Now, if we call type.__new__ with a valid subclass of type as first parameter:

In [13]: class InheritFromOtherTake2(object):
    ...:     def __new__(mcls, name, bases, ns):
    ...:         return type.__new__(type, name, bases, ns)
    ...:     

In [14]: class test5(metaclass=InheritFromOtherTake2):
    ...:     pass
    ...: 

In [15]: type(test5)
Out[15]: type

Just for completeness, as mentioned above, it is indeed possible for the callable used as metaclass return something other than an instance of type (or a subclass of it). In that case, the object resulting from the class statement body simply won't be a class, but whatever that callable returned:

In [7]: def dict_maker_meta(name, bases, ns):
   ...:     return ns
   ...: 

In [8]: class test3(metaclass=dict_maker_meta):
   ...:     a = 1
   ...:     b = 2
   ...:     c = 'test'
   ...:     

In [9]: type(test3)
Out[9]: dict

In [10]: test3
Out[10]: 
{'__module__': '__main__',
 '__qualname__': 'test3',
 'a': 1,
 'b': 2,
 'c': 'test'}