Remove empty dicts in nested dictionary with recursive function

Question

I'm trying to remove non-values from a nested dictionary. My first effort works fine, but unfortunately keys pointing to now empty dicts still persist.

So if i do:

pass1 = stripper(my_dict)
return stripper(pass1)

This works, but i'm thinking a more elegant nested solution might be possible?

def stripper(self, data):
    if isinstance(data, dict):
        d = ({k: stripper(v) for k, v in data.items()
             if v not in [u'', None]})
        if d:
            return d
    else:
        return data

Edit:

Failing example, dict below returns as {'foo': 'bar', 'bar': None}:

{
    'foo': 'bar',
    'bar': {
        'foo': None,
        'one': None
    }
}

Answer 1

The dict comprehension is certainly concise but if you expand it out, the solution becomes more obvious:

def stripper(self, data):
    new_data = {}
    for k, v in data.items():
        if isinstance(v, dict):
            v = stripper(v)
        if not v in (u'', None, {}):
            new_data[k] = v
    return new_data

Answer 2

To extend the accepted answer to include objects that might be an element of a list:

def clean_empties(value: dict | list) -> dict | list:
    if isinstance(value, dict):
        value = {
            k: v
            for k, v in ((k, clean_empties(v)) for k, v in value.items())
            if v
        }
    elif isinstance(value, list):
        value = [v for v in (clean_empties(v) for v in value) if v]
    return value
        

# Usage:
data = {
    "None": None,
    "empty_list": [],
    "empty_dict": {},
    "empty_string": "",
    "list_with_empties": [
        "",
        {},
        [],
        None,
        {
            "more_empties": "",
            "and_even_more": [""],
            "one_thing_i_care_about": "hi",
        },
    ],
}
# Or alternatively data can be a list
# data = ['', {}, [], None, {'more_empties': '', 'and_even_more': [''], 'one_thing_i_care_about': 'hi'}]
cleaned = clean_empties(data)
print(cleaned)

Answer 3

Answer of @Oliver is correct, but having some edge cases checks won't hurt, here you go: (Was unable to edit Oliver's answer as the queue was full)

    def dictionary_stripper(data):
        new_data = {}
        
        # Only iterate if the given dict is not None
        if data:
            for k, v in data.items():
                if isinstance(v, dict):
                    v = dictionary_stripper(v)
                
                # ideally it should be not in, second you can also add a empty list if required
                if v not in ("", None, {}, []):
                    new_data[k] = v
            
            # Only if you want the root dict to be None if empty
            if new_data == {}:
                return None
            return new_data
        return None

Answer 4

I landed here with a need to drop None values yet preserve empty dicts and lists. Reused/modified @OliverDain answer, thanks for that. Hope this helps someone.

def drop_none_values(data):
    """
    Recursively remove None values in a dictionary or list;
    this includes list of dicts, etc. Do not drop empty
    dicts or lists.

    :param data: Object to process for None values.
        Returns a new dict if passed a dict;
        returns a new list if passed a list;
        returns the argument unchanged otherwise.
    """
    if isinstance(data, dict):
        new_data = {}
        for k, v in data.items():
            v = drop_none_values(v)
            if v is not None:
                new_data[k] = v
    elif isinstance(data, list):
        new_data = []
        for v in data:
            v = drop_none_values(v)
            if v is not None:
                new_data.append(v)
    else:
        new_data = data
    return new_data

Here's my test case, that function above is in a file/module named util:

def test_drop_none_values():
    data = {
        'first': None,
        'second': None,
        'empty': [],
        'alist': ['string', None, {'foo': None, 'bar': 'baz'}]
    }
    assert len(data) == 4
    stripped = util.drop_none_values(data)
    assert len(stripped) == 2
    assert len(stripped['alist']) == 2
    assert len(stripped['alist'][1]) == 1