Appending dataframe to empty dataframe is not working

Question

I declared an empty data frame near the top of my file with a global scope:

final_df = pd.DataFrame()

I have stats_dfsuccessfully printing the correct value, but final_df is not changing after appending stats_df to it:

stats_df = pd.DataFrame(X, columns=stats_feature_names).sum().to_frame().T
print('statsdf being appended: \n', stats_df)
print('final_df before append: \n', final_df)
final_df.append(stats_df)
print('final_df after append: \n', final_df)

The output of these print statements are:

statsdf being appended: 
    GF  GA
0  14  33
final_df before append: 
 Empty DataFrame
Columns: []
Index: []
final_df after append: 
 Empty DataFrame
Columns: []
Index: []

When it should be:

statsdf being appended: 
    GF  GA
0  14  33
final_df before append: 
 Empty DataFrame
Columns: []
Index: []
final_df after append: 
 GF  GA
0  14  33

Why is stats_df not being appended to final_df?

jezrael · Accepted Answer · 2018-03-24 12:06:05Z

5

You need assign to new DataFrame, because use DataFrame.append, not pure python append:

stats_feature_names = ['a','b']
final_df = pd.DataFrame()

X = [[1,2]]
stats_df = pd.DataFrame(X, columns=stats_feature_names).sum().to_frame().T
print('statsdf being appended: \n', stats_df)
print('final_df before append: \n', final_df)
final_df = final_df.append(stats_df, ignore_index=True)
print('final_df after append: \n', final_df)
    a  b
0  1  2

But better solution is append to list (pure python append) and out of loop use concat:

L = []
for x in iterator:
    stats_df = pd.DataFrame([[1,2]], columns=stats_feature_names).sum().to_frame().T
    L.append(stats_df)

final_df = pd.concat(L, ignore_index=True)
print('final_df after append: \n', final_df)

edited Mar 24, 2018 at 12:06

answered Mar 24, 2018 at 11:27

jezrael

868k102 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

tom44 Over a year ago

That works, thank you! Only problem is that when I use this, the index in the data frame is not incrementing: final_df after append: GF GA 0 17 27 0 15 27 0 14 30 0 14 33

jezrael Over a year ago

You need , ignore_index=True), check edited answer.

Collectives™ on Stack Overflow

Appending dataframe to empty dataframe is not working

1 Answer 1

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Linked

Related