dataframe.replace() within a for loop

Question

I'm trying to replace values in num column. For each letter in ab, I have a dictionary. I've shown only two below (A & B)

data = pd.DataFrame( {'ab' : ['A','B','A','A','B'], 'num' : ['01','02','01','01','01']})

a_replacements = { 'num' : { '01' : 'funny', '02' : 'serious' }}
b_replacements = { 'num' : { '01' : 'beginning', '02' : 'end' }}

data[data.ab == 'A'] = data[data.ab == 'A'].replace(inplace=True, to_replace=a_replacements)

The assignment in the last line works fine. But when I try to use this inside of a for loop, where I have to replaced values in num for 26 different letters in ab, I face the following issue:

for letter in data.ab.unique():
    data.loc[data.ab == letter] = data.replace(to_replace=letter.lower()+"_replacements")

To which I get:

---

TypeError                                 Traceback (most recent call last)
<ipython-input-96-acd3197ceef4> in <module>()
      1 for letter in data.ab.unique():
      2     print(letter.lower()+"_replacements")
----> 3     data.loc[data.ab == letter] = data.replace(to_replace=letter.lower()+"_replacements")

/Users/alokshenoy/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/pandas/core/generic.py in replace(self, to_replace, value, inplace, limit, regex, method, axis)
   3427             if isinstance(to_replace, (tuple, list)):
   3428                 return _single_replace(self, to_replace, method, inplace,
-> 3429                                        limit)
   3430 
   3431             if not is_dict_like(to_replace):

/Users/alokshenoy/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/pandas/core/generic.py in _single_replace(self, to_replace, method, inplace, limit)
     70     if self.ndim != 1:
     71         raise TypeError('cannot replace {0} with method {1} on a {2}'
---> 72                         .format(to_replace, method, type(self).__name__))
     73 
     74     orig_dtype = self.dtype

TypeError: cannot replace ['a_replacements'] with method pad on a DataFrame

Any ideas on how to solve this?

Answer 1

You can use groupby with apply for replace by dict of all dicts created by zip which map list of names in column ab to list of dicts:

a_replacements = { 'num' : { '01' : 'funny', '02' : 'serious' }}
b_replacements = { 'num' : { '01' : 'beginning', '02' : 'end' }}
abnames = ['A','B']
L = [a_replacements, b_replacements]

replacements = dict(zip(abnames, L))
print (replacements)
{'A': {'num': {'01': 'funny', '02': 'serious'}}, 
'B': {'num': {'01': 'beginning', '02': 'end'}}}

df = data.groupby('ab').apply(lambda x: x.replace(replacements[x.name]))
print (df)
  ab        num
0  A      funny
1  B        end
2  A      funny
3  A      funny
4  B  beginning

Answer 2

The solution to your problem is in groupby. This will allow you to process each group that uses unique replacements.

Key Code:

for name, group in data.groupby(['ab']):
    data[data.ab == name] = group.replace(to_replace=replacements[name])

All Code:

import pandas as pd

data = pd.DataFrame({'ab': ['A', 'B', 'A', 'A', 'B'],
                     'num': ['01', '02', '01', '01', '01']})
print(data)

replacements = dict(
    A={'num': {'01': 'funny', '02': 'serious'}},
    B={'num': {'01': 'beginning', '02': 'end'}},
)

for name, group in data.groupby(['ab']):
    data[data.ab == name] = group.replace(to_replace=replacements[name])

print(data)

Results:

  ab num
0  A  01
1  B  02
2  A  01
3  A  01
4  B  01

  ab        num
0  A      funny
1  B        end
2  A      funny
3  A      funny
4  B  beginning