Dynamically creating a class from file

Question

I've seen these "Dynamically create a class" questions which are answered saying, "use the type() function". I'm sure I'll have to at some point but right know I'm clueless. But from what I've seen you have to already know something about the class, such as a name.

What I'm trying to do is parse an idl type of file and from that create a class which will have methods and attributes. So I have NO knowledge up front of what the class name, functions, arguments or anything will be until I parse the string.

Any ideas?

Answer 1

http://docs.python.org/library/functions.html#type

It's a bit hard to Google for, but you can search for python type(name, bases, dict) function examples to get:

http://www.voidspace.org.uk/python/articles/metaclasses.shtml

An excerpt from the above, which gets to the heart of your question:

---

The following are basically equivalent:

def __init__(self, x):
    self.x = x

def printX(self):
    print self.x

Test = type('Test', (object,), {'__init__': __init__, 'printX': printX})

and:

class Test(object):
    def __init__(self, x):
        self.x = x

    def printX(self):
        print self.x

---

There are two ways to create functions on the fly that I can think of. The usually-bad way is to write the code and reparse it (though done correctly, this can greatly increase performance). The sane way is to implement a function which interprets your IDL. This is called a higher-order function: http://effbot.org/pyfaq/how-do-you-make-a-higher-order-function-in-python.htm

An example of what you would write, if you cannot find an interpreter for your IDL (of if it's a custom IDL) is something like the link above, such as:

def makeMethod(idlCode):
    syntax = MyIDL.parse(idlCode)

    def newMethod(*args, **kw):
        if syntax.statementType == MyIDL.IF_STATEMENT:
            if secureLookup(mySyntaxTree.IF):
               return secureLookup(args[0]) 
            else:
               return secureLookup(args[1])
        ...

    return (syntax.methodName, newMethod)

There are many more elegant ways to expand this method, if you set up a mapping between constructs of your IDL and the syntax of *args and **kw, but this gives you the most flexibility and is the most straightforward and basic way I could think of.

Then you'd do pass in:

class DynamicIdlClass(object):
    ...

for idlObject in idlCode:
    methods = dict(makeMethod(clause) for clause in idlObject.clauses})
    methods['__init__'] = makeInitMethod(idlObject.initClause)
    idlObject = type('Test', (DynamicIdlClass,), methods)

    yield idlObject  # or idlObjectsList.push(idlObject), etc.

Answer 2

On our project we create an object that exposes a concrete method that has the ability to add callable attributes. We parse a lot of yaml files for the schema and assemble these objects dynamically.

Something similar to

def add_method(name, callable):
    setattr(self,name,callable)

But this is a very naive example but you get the idea. The schema is stored inside each dynamic class under something like __schema which is a dictionary containing kwarg prototypes and docstrings.

For the actual callables make a skeleton like

def skel(self, *args, **kwargs):
    if '_schema' in kwargs:
        parse_yml, add control statements
    handle args and kwargs based on schema

some_instance.add_method('blah',skel)

Again, these are both extremely naive examples and do not cover the majority of issues and edge cases which arise when working with this type of problem. This is also one of several methods to find a solution.

Examples:

IonObject

Object Model Generator

Answer 3

Rather than thinking of this as 'how do I dynamically create a class', think of it as an opportunity to use the metaprogramming capabilities of Python instead.

The first place I'd look for ideas is to the standard library source for xmlrpclib -- look specifically at how it uses __getattr__ to handle otherwise non-existent method calls on the fly.

I know that I didn't really understand why they call Python a dynamic language until I had fully come to terms with those possibilities.