12

I'm trying to write a Database Abstraction Layer in Python which lets you construct SQL statments using chained function calls such as:

results = db.search("book")
          .author("J. K. Rowling")
          .price("<40.00")
          .title("Harry")
          .execute()

but I am running into problems when I try to dynamically add the required methods to the db class.

Here is the important parts of my code:

import inspect

def myName():
    return inspect.stack()[1][3]

class Search():

    def __init__(self, family):
        self.family = family
        self.options = ['price', 'name', 'author', 'genre']
        #self.options is generated based on family, but this is an example
        for opt in self.options:
            self.__dict__[opt] = self.__Set__
        self.conditions = {}

    def __Set__(self, value):
        self.conditions[myName()] = value
        return self

    def execute(self):
        return self.conditions

However, when I run the example such as:

print(db.search("book").price(">4.00").execute())

outputs:

{'__Set__': 'harry'}

Am I going about this the wrong way? Is there a better way to get the name of the function being called or to somehow make a 'hard copy' of the function?

Improve this question
3
  • 2
    May I ask why you are trying to reinvent SQLAlchemy? Commented Nov 26, 2011 at 6:42
  • 6
    I actually quite commonly try to code my own libraries which replicate those that already exist so that I can learn more about the language and get a better feel for how the more advanced parts come together :) Commented Nov 26, 2011 at 6:44
  • 5
    OK, a learning exercise, that is a good reason. Although in this case I think reading the SQLAlchemy source code will be a good start. ORM's are complex and magical. Commented Nov 26, 2011 at 6:59

3 Answers 3

Reset to default
13

You can simply add the search functions (methods) after the class is created:

class Search:  # The class does not include the search methods, at first
    def __init__(self):
        self.conditions = {}

def make_set_condition(option):  # Factory function that generates a "condition setter" for "option"
    def set_cond(self, value):
        self.conditions[option] = value
        return self
    return set_cond

for option in ('price', 'name'):  # The class is extended with additional condition setters
    setattr(Search, option, make_set_condition(option))

Search().name("Nice name").price('$3').conditions  # Example
{'price': '$3', 'name': 'Nice name'}

PS: This class has an __init__() method that does not have the family parameter (the condition setters are dynamically added at runtime, but are added to the class, not to each instance separately). If Search objects with different condition setters need to be created, then the following variation on the above method works (the __init__() method has a family parameter):

import types

class Search:  # The class does not include the search methods, at first

    def __init__(self, family):
        self.conditions = {}
        for option in family:  # The class is extended with additional condition setters
            # The new 'option' attributes must be methods, not regular functions:
            setattr(self, option, types.MethodType(make_set_condition(option), self))

def make_set_condition(option):  # Factory function that generates a "condition setter" for "option"
    def set_cond(self, value):
        self.conditions[option] = value
        return self
    return set_cond

>>> o0 = Search(('price', 'name'))  # Example
>>> o0.name("Nice name").price('$3').conditions
{'price': '$3', 'name': 'Nice name'}
>>> dir(o0)  # Each Search object has its own condition setters (here: name and price)
['__doc__', '__init__', '__module__', 'conditions', 'name', 'price']

>>> o1 = Search(('director', 'style'))
>>> o1.director("Louis L").conditions  # New method name
{'director': 'Louis L'}
>>> dir(o1)  # Each Search object has its own condition setters (here: director and style)
['__doc__', '__init__', '__module__', 'conditions', 'director', 'style']

Reference: http://docs.python.org/howto/descriptor.html#functions-and-methods

If you really need search methods that know about the name of the attribute they are stored in, you can simply set it in make_set_condition() with

       set_cond.__name__ = option  # Sets the function name

(just before the return set_cond). Before doing this, method Search.name has the following name:

>>> Search.price
<function set_cond at 0x107f832f8>

after setting its __name__ attribute, you get a different name:

>>> Search.price
<function price at 0x107f83490>

Setting the method name this way makes possible error messages involving the method easier to understand.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

+1. You're right. I was being too clever. There's nothing special about these condition setter methods that requires them to be created on each instance.
I was assuming these methods where to be be set automatically from the schema, maybe I was assuming too much. :-)
According to the problem description the method names will vary based on the family, so setting the names on the class will result in either missing names for some families, or all the names for every family.
@EthanFurman: You're right. I guessed (perhaps incorrectly) that family was actually not a proper __init__() parameter because __init__() was always called with the same family, in a given program. I added a PS that stresses this point and gives a solution that gives each Search instance its own family of possible option searches.
5

Firstly, you are not adding anything to the class, you are adding it to the instance.

Secondly, you don't need to access dict. The self.__dict__[opt] = self.__Set__ is better done with setattr(self, opt, self.__Set__).

Thirdly, don't use __xxx__ as attribute names. Those are reserved for Python-internal use.

Fourthly, as you noticed, Python is not easily fooled. The internal name of the method you call is still __Set__, even though you access it under a different name. :-) The name is set when you define the method as a part of the def statement.

You probably want to create and set the options methods with a metaclass. You also might want to actually create those methods instead of trying to use one method for all of them. If you really want to use only one __getattr__ is the way, but it can be a bit fiddly, I generally recommend against it. Lambdas or other dynamically generated methods are probably better.

Improve this answer

7 Comments

Here is the reference about __*__ names: docs.python.org/reference/…
@EOL I've always thought of 'xxx' names as 'private' properties or methods of a class that don't/can't get called from outside of it? Is there any convention which you are supposed to use for these?
@Ben: Yes, there is a convention, which is very similar to the notation reserved by Python: it's __xxx (no trailing "dunder")–same reference as above.
@EOL: No, that's not a convention, that's what triggers name mangling. Different.
@Ben: The convention to mark something as internal is to start it with one underscore. Having two underscores triggers a name mangling that can be useful to avoid naming clashes.
|
5

Here is some working code to get you started (not the whole program you were trying to write, but something that shows how the parts can fit together):

class Assign:

    def __init__(self, searchobj, key):
        self.searchobj = searchobj
        self.key = key

    def __call__(self, value):
        self.searchobj.conditions[self.key] = value
        return self.searchobj

class Book():

    def __init__(self, family):
        self.family = family
        self.options = ['price', 'name', 'author', 'genre']
        self.conditions = {}

    def __getattr__(self, key):
        if key in self.options:
            return Assign(self, key)
        raise RuntimeError('There is no option for: %s' % key)

    def execute(self):
        # XXX do something with the conditions.
        return self.conditions

b = Book('book')
print(b.price(">4.00").author('J. K. Rowling').execute())
Improve this answer

3 Comments

Each and every search creates an Assign instance, which is a bit time and memory heavy. :) Is there any advantage of this approach compared to the direct "add methods after class creation" method outlined in my answer?
@EOL This approach is how Python itself works and it is a common idiom. For example, Python creates a new BoundMethod instance every time you make a method call such as a.m(x). The OP is trying to learn how Python works and it is appropriate to teach __call__ for calls and __getattr__ for dynamic dotted lookup. That is what they are for.
Thanks. Yes, __getattr__ and __call__ are indeed important to know when it comes to dynamic lookup. I'm not sure this is what the OP needs, though: I understand that he wants to define non-dynamical class attributes, instead of dynamical instance attributes. No? (That pretty much sums up the two different approaches taken in our answers, I think.)

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.