Python enconding with mysql

Question

I'm using an Active Record called Storm in my Python code to retrieve some records on a MySQL database.

The problem is: My table is in 'utf8_unicode_ci', but when I retrieve the object I get 'latin-1' attributes, so I need to do object.attr.decode('latin-1').encode('utf-8') wich isn't working always - throwing some exceptions.

My question: This is a python behavior? a MySQL behavior? Something related to the Storm?

The code:

Storm.conn(user=db_user,db=db_name, passwd=db_passwd)
events = Event.select('*',status='=2',date_end='>=NOW()')
for event in events:
    now = datetime.datetime.now().strftime("%Y-%m-%d %H:%M")
    try:
        #here we need only utf-8 strings
        conn.index({"title": event.get_title(), "local": event.get_local(), "url": event.getUrl(), "description": event.get_description(), "artists": event.get_artists(), "tags": event.get_tags(), "picture": event.get_Picture(), "type": event.get_type(), 'date_begin': event.get_date_begin(), 'date_end': event.get_date_end(), '_ttl': event.get_ttl()}, "wf", "event", event.id)
        print 'Indexed - '+now+': '+str(event.id)
    except Exception,error:
        print 'Error - '+now+': '+str(event.id)+" - "+str(error)

"#here we need only utf-8 strings" You just lost the game. Always use unicodes. — Ignacio Vazquez-Abrams
– Ignacio Vazquez-Abrams, Commented Aug 31, 2012 at 18:15
@IgnacioVazquez-Abrams UTF-8 isn't Unicode? en.wikipedia.org/wiki/Unicode — tfcstack
– tfcstack, Commented Aug 31, 2012 at 18:19
No, it isn't. And it especially has nothing to do with unicode. — Ignacio Vazquez-Abrams
– Ignacio Vazquez-Abrams, Commented Aug 31, 2012 at 18:23
@Ignacio is being cryptic: In python, unicode is a type of text object, utf-8 is a format for saving ("encoding") text to disk. Read python's unicode documentation. — alexis
– alexis, Commented Sep 1, 2012 at 21:15

alexis · Accepted Answer · 2012-09-01 21:22:43Z

2

No idea about the specifics of your stack, but with MySQL you need to set the encoding of the connection separately from the encoding of the tables. I had some unicode tables and it took me a long time to realize that the connection was set to latin-1, so my unicode data got interpreted as Latin-1 bytes and "converted" to nonsense unicode at the far end.

answered Sep 1, 2012 at 21:22

alexis

50.4k18 gold badges107 silver badges172 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

tfcstack Over a year ago

Indeed it worked :D changing from: Storm.conn(user=db_user,db=db_name, passwd=db_passwd) to Storm.conn(user=db_user,db=db_name, passwd=db_passwd, charset = "utf8", use_unicode = True) made it work. Thanks alexis :D

kisileno · Accepted Answer · 2012-08-31 18:31:35Z

1

Maybe this answer seems stupid, but try to print this in the top of your py file after interpreter path

#!/usr/bin/env python
# -*- coding: utf-8 -*-

answered Aug 31, 2012 at 18:31

kisileno

7879 silver badges23 bronze badges

1 Comment

tfcstack Over a year ago

There is no such thing as stupid answer :D But our code already have this head. Thanks anyway.

Collectives™ on Stack Overflow

Python enconding with mysql

2 Answers 2

1 Comment

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Related