Update Dataframe while iterating through it - Python

Question

This is my first time working with DataFrame and I am badly stuck on a point where I need to Update DataFrame.

My dataframe object has some basic values like userId, userName, address and then I need to call a model that returns a list of values (Currently 36 values). I need to append that values in existing dataframe object. Either in single cell or in multiple cells in same row.

But I'm getting different errors while trying different methods.

This is my current Dataframe:

UserId UserName Address 
1      User1    Add1
2      User2    Add2

This is what I want to do:

UserId UserName Address Res
1      User1    Add1    [1,2,3...36]
2      User2    Add2    [1,2,3...36]

Or something like this:

UserId UserName Address Res1 Res2 Res3... Res36
1      User1    Add1    1      2    3 ... 36
2      User2    Add2    1      2    3 ... 36

Currently I'm using this method..

for index, row in rawData.iterrows():
    res = calculateData(row)
    rawData['res'] = pd.Series(res, index=rawData.index)

It throws this error:

ValueError: Length of passed values is 36, index implies 96.

On debugging, I got to know that, it is trying to add all values (36) for each row (96) in my dataframe while I'm trying to add it in a cell of index. I did try some different methods as well but All I got is length error because every method adds list to each row..

Any help on how can I achieve my required result?

Answer 1

Use list comprehension on dataframe index and call function calculateData inside list comprehension.

Assume your function calculateData is defined as follows:

def calculateData(x):
    return np.arange(36) + x.name


rawData['res'] = [calculateData(rawData.loc[i]) for i in rawData.index]

Out[20]:
   UserId UserName Address                                                res
0       1    User1    Add1  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,...
1       2    User2    Add2  [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14...