What is efficient way of removing empty values from dict inside list?

Question

What is efficient way of removing empty values from dictionary which is saved inside the list.

list1 = [{'l1k1': 'l1v1', 'l1k2': 'l1v2'}, {'l2k1': 'l2v1', 'l2k2': ''}]
list2 = []
for l in list1:
  d = {}
  for k, v in l.items():
    if v.strip() is not None or v.strip() != '':
      d[k] = v
  list2.append(d)
print(list2)

Actual Output:

[{'l1k1': 'l1v1', 'l1k2': 'l1v2'}, {'l2k1': 'l2v1', 'l2k2': ''}]

Expected Output:

[{'l1k1': 'l1v1', 'l1k2': 'l1v2'}, {'l2k1': 'l2v1'}]

picky, but in your example you actually do not remove the key/value pairs, you just don't copy them — ezdazuzena
– ezdazuzena, Commented Feb 25, 2019 at 13:12
Suppose v is an empty string. In that case (v.strip() is not None) will be True, so the empty string is added to the dict d anyway due to the 'or' condition. Reasoning about combining negatives (is not and !=) is hard and I'd recommend avoiding it. — Simon Hibbs
– Simon Hibbs, Commented Feb 25, 2019 at 13:15

Arkistarvh Kltzuonstev · Accepted Answer · 2019-02-25 13:29:30Z

9

Try this :

list2 = [{k:v for k,v in i.items() if v!= '' or v.strip() != ''} for i in list1]

We can use dict-comprehension as well as list-comprehension simultaneously. We loop for every element (which is a dictionary) of list1, and just take those key-value pairs where the value or value.strip() for the corresponding key in the dictionary is not a vacant string.

A shorter version for cancelling the values with None type also:

list2 = [{k:v for k,v in i.items() if v} for i in list1]

edited Feb 25, 2019 at 13:29

answered Feb 25, 2019 at 13:14

Arkistarvh Kltzuonstev

6,9747 gold badges32 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Nullman Over a year ago

this is good but you should probably explain a little what it does, comprehensions aren't the most obvious things to new comers

Óscar López Over a year ago

This is too verbose (not to mention mistaken, because of the or), and also it's not handling the None case. Check my answer for a simpler solution.

khelwood Over a year ago

The strip() part in this code is useless, because you're using or. If v=' ' then v!='' or v.strip()!='' is true.

Óscar López · Accepted Answer · 2019-02-25 13:23:53Z

7

Try this:

list1 = [{'l1k1': 'l1v1', 'l1k2': 'l1v2'}, {'l2k1': 'l2v1', 'l2k2': ''}]
list2 = [{ k: v for k, v in d.items() if v and v.strip() } for d in list1]

Notice that the correct check to do here is v and v.strip(), that ensures that the string is not None and is not all spaces. It works as expected:

list2
=> [{'l1k1': 'l1v1', 'l1k2': 'l1v2'}, {'l2k1': 'l2v1'}]

It's efficient because it uses list comprehensions and dictionary comprehensions, which are faster than doing explicit loops. Also, it's quite compact and idiomatic :)

edited Feb 25, 2019 at 13:23

answered Feb 25, 2019 at 13:13

Óscar López

237k38 gold badges320 silver badges391 bronze badges

3 Comments

Palla veera Over a year ago

I do not know, I am getting same result, list1 = [{'l1k1': 'l1v1', 'l1k2': 'l1v2'}, {'l2k1': 'l2v1', 'l2k2': ''}] list2 = [ { k: v for k, v in d.items() } for d in list1] print(list2) print(list1)

Palla veera Over a year ago

Sometimes i get this error: AttributeError: 'dict' object has no attribute 'strip'

Óscar López Over a year ago

@Pallaveera that means that some of your dicts have values that are dicts themselves! Check your input data, it doesn't look like the one in the question.

Rustang · Accepted Answer · 2019-02-25 13:17:17Z

0

Here is a simple alternative, not using list comprehension that may be easier for you to follow:

for d in list1:
  for k in d.keys():
      if d[k] != '':
          list2.append({k:d[k]})

answered Feb 25, 2019 at 13:17

Rustang

5465 silver badges17 bronze badges

Comments

Nullman · Accepted Answer · 2019-02-25 13:23:47Z

making a copy of your list is not very efficient, unless you WANT to keep the original as well. you can just remove the entries that are "empty" like so:

list1 = [{'l1k1': 'l1v1', 'l1k2': 'l1v2'}, {'l2k1': 'l2v1', 'l2k2': ''}]
for item in list1:
    for key in list(item.keys()):
        if not item[key].strip():
            del item[key]

this part list(item.keys()) is important. why list? because python doesnt like it when you change the size the dict during iteration, if we pre-get the keys, we will be iterating on a list instead of a generator.

not item[key].strip() is also a good to note, empty strings evaluate to False while non-empty string evaluate to True

*note: strictly speaking the .keys() isnt necessary, but it does help readability if youre new to python

tuomastik · Accepted Answer · 2020-11-25 07:59:15Z

list1 = [{'l1k1': 'l1v1', 'l1k2': 'l1v2'}, {'l2k1': 'l2v1', 'l2k2': ''}]

The most efficient way is this:

list2 = [{k: v for k,v in dic.items() if v} for dic in list1]

PS:

using .strip() works for strings only and since you have all types of False/None/"" values those would not be removed, infact you would get bugs. Also: using .strip() just to remove " " (with a space) is wrong since " " is not even a False value in Python and you really should not consider it as such also. To remove certain characters (other than empty ones as in your question) you should try to create and use a different specific function instead that does just that.

Collectives™ on Stack Overflow

What is efficient way of removing empty values from dict inside list?

5 Answers 5

3 Comments

3 Comments

Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

3 Comments

3 Comments

Comments

Comments

Comments

Related