Pandas dataframe to_sql with data longer than 65536 characters

Question

I have a Pandas dataframe, where some columns have values longer than 65536 characters. When I tried to export the data to MySQL using df.to_sql(con=engine, name=table_name, if_exists='replace', index=False), they were truncated to 65536 characters.

Is there a way to automatically convert a column to LONGTEXT or BLOB (instead of TEXT) if it has values longer than 65536 so that the table content won't be truncated?

yuan0122 · Accepted Answer · 2019-07-10 23:10:39Z

2

This might be a workaround. The only thing is you need to have the list of columns that need to be converted to LONGTEXT.

from sqlalchemy.dialects.mysql import LONGTEXT
dtype = {
    "long_column_1": LONGTEXT,
    "long_column_2": LONGTEXT
}
pdf.to_sql(con=engine, name=table_name, if_exists='replace', index=False, dtype=dtype)

answered Jul 10, 2019 at 23:10

yuan0122

4379 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Pandas dataframe to_sql with data longer than 65536 characters

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related