Dictionary Parser Class

Question

I'm working on a parser library for substituting strings in dictionaries, with things like globals etc. Wanting to get opinions on the direction I'm taking and where I can improve the script. This currently works

class DictParser(object):
    def __init__(self, dictionary):
        self.working_dict = dictionary

    def __traverse(self, obj, callback=None):
        if isinstance(obj, dict):
            return {k: self.__traverse(v, callback) for k, v in obj.items()}
        elif isinstance(obj, list):
            return [self.__traverse(elem, callback) for elem in obj]
        else:
            if not callback:
                return obj
            else:
                return callback(obj)

    def replace_with_global_var(self, pattern):
        pattern_variable = re.compile(pattern)

        def substitute(x):
            if x.group(1) in globals():
                return globals()[x.group(1)]
            else:
                raise IndexError('Missing Global: %s' % x.group(1))

        def transformer(item):
            if isinstance(item, str):
                return pattern_variable.sub(substitute, item)
            else:
                return item

        self.working_dict = self.__traverse(self.working_dict, callback=transformer)

    def retrieve(self):
        return self.working_dict

Usage:

a = 'blah'

my_dict = {
    'test': '((a))',
    'another': 'my ((a)) test',
    'nested': {
        'child': 'here is ((a)) ((a))'
    },
    'boo': 1,
    'test': False
}

d = DictParser(my_dict)
d.replace_with_global_var(r'\(\((.+?)\)\)')
print(d.retrieve())

This was a reasonably quick hack together, tried using repl so I could print the d variable directly but didn't quite work out.

Community · Accepted Answer · 2017-05-23 12:41:00Z

I would have used str.format together with a transparent dictionary to achieve this result. It gives you the familiarity of the format syntax and not having to come up with the regex on your own. This comes at the cost of not being able to specify the regex anymore at all.

The transparent dict is taken from this answer and is needed because otherwise format raises a KeyError when trying to do a partial replace.

from collections import defaultdict

class SafeDict(dict):
    def __missing__(self, key):
        return '{' + key + '}'


def parse(val, vars, callback=None):
    if callback:
        val = callback(val)
    if isinstance(val, dict):
        return parse_dict(val, vars, callback)
    elif isinstance(val, list):
        return parse_list(val, vars, callback)
    try:
        return val.format(**SafeDict(vars))
    except Exception:
        return val


def parse_list(l, vars, callback=None):
    parsed = type(l)()
    for val in l:
        parsed.append(parse(val, vars, callback))
    return parsed


def parse_dict(d, vars, callback=None):
    parsed = type(d)()
    for key, val in d.items():
        parsed[key] = parse(val, vars, callback)
    return parsed

parse_dict and parse_list could be replaced by the one-liners you use. This way, however, they return the same data type as was originally there (this is relevant for sub-classes of list or dict). If you don't need this, just replace them with what you've got.

usage:

a = 'blah' 
my_dict = {'test': '{a}', 
           'another': 'my {a} test',
           'nested': {'child': 'here is {a} {a}'},
           'boo': 1,
           'test2': False}
print(parse(my_dict, globals()))

When adding more parsers for other data types than list or dict (their sub-classes are already included), you would probably define a mapping from type to parsers to avoid repetition in parse.

As an alternative you should have a look at f-strings, which will be included in Python 3.6, scheduled to be released soon.

With this you could do:

a = 'blah' 
table = "Users"
row = 42
parsed_dict = {'test': f'{a}', 
               'another': f'my {a} test',
               'nested': {'child': f'here is {a} {a}'},
               'boo': 1,
               'test2': False,
               'complex': f'result: {db_lookup(table, row))}'}

Unfortunately that'd be quite limiting, I need to use this with multiple replace functions, not just pthe variable one. I plan on extending it to do things like looking up the value of a DB table etc (i.e. $tablename:column:filter$) so the regex' will be vital I think. — Allan M
– Allan M, Commented Oct 20, 2016 at 6:58
@AllanM You should have a look at f-strings, which will be included in Python 3.6. They allow general expressions inside the {}. — Graipher
– Graipher, Commented Oct 20, 2016 at 7:04
Unfortunately I need to support Python 2 so that's not gonna be an option. I'll have a play with your suggestion. — Allan M
– Allan M, Commented Oct 20, 2016 at 7:07
@Graipher Except f-strings need that the expression within the curly braces is valid at the time the f-string is defined. This can be a limitation if you want to build a query based on the variables you want to replace. Instead of fetching the whole DB beforehand. — 301_Moved_Permanently
– 301_Moved_Permanently, Commented Oct 20, 2016 at 7:11
@AllanM Yes, it definitely is. And there is actually already a solution to this, i-strings. They are just like f-strings but delay the evaluation by allowing you to specify your own __format__ function. But that won't come till a later Python version and is irrelevant if you need to support Python 2.x. — Graipher
– Graipher, Commented Oct 20, 2016 at 7:13

Stack Exchange Network

Dictionary Parser Class

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Dictionary Parser Class

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions