Pandas concatenate and merge two dataframes

Question

I have two dataframes that have mostly different columns, but two of the columns are almost the same, frame and date.

df_1 
id  FRAME  var_1    date
 1     10     15  3/4/16
 2     12     69  3/5/17

df_2
id  frame  var_2        date_time
 1     11     15  3/2/16 08:14:32
 2     12     69  3/5/17 09:12:29

Right now, I'm using pd.concat as df_3 = pd.concat([df_1, df_2], axis=0, ignore_index=True)

df_3 
id  FRAME  var_1    date  frame   var_2       date_time
 1     10     15  3/4/16    NaN    NaN              NaT
 2     12     69  3/5/17    NaN    NaN              NaT
 3    NaN    NaN     NaT     11     15  3/2/16 08:14:32
 4    Nan    NaN     NaT     12     69  3/5/17 09:12:29

What I would like to have is the FRAME and date/date_time columns merged

df_3 
id  FRAME  var_1   var_2       date_time
 1     10     15     NaN           3/4/16
 2     12     69     NaN           3/5/17
 3     11    NaN      15  3/2/16 08:14:32
 4     12    NaN      69  3/5/17 09:12:29

Erfan · Accepted Answer · 2019-08-29 23:20:46Z

2

Use pd.concat with rename:

df_3 = pd.concat([df_1, 
                  df_2.rename(columns={'frame':'FRAME', 'date_time':'date'})], 
                 ignore_index=True, 
                 sort=True)

Output

   FRAME             date  var_1  var_2
0     10           3/4/16   15.0    NaN
1     12           3/5/17   69.0    NaN
2     11  3/2/16 08:14:32    NaN   15.0
3     12  3/5/17 09:12:29    NaN   69.0

answered Aug 29, 2019 at 23:20

Erfan

43.3k9 gold badges75 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Pandas concatenate and merge two dataframes

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related