Python: Using .format() on a Unicode-escaped string

Question

I am using Python 2.6.5. My code requires the use of the "more than or equal to" sign. Here it goes:

>>> s = u'\u2265'
>>> print s
>>> ≥
>>> print "{0}".format(s)
Traceback (most recent call last):
     File "<input>", line 1, in <module> 
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2265'
  in position 0: ordinal not in range(128)`  

Why do I get this error? Is there a right way to do this? I need to use the .format() function.

Answer 1

Just make the second string also a unicode string

>>> s = u'\u2265'
>>> print s
≥
>>> print "{0}".format(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2265' in position 0: ordinal not in range(128)
>>> print u"{0}".format(s)
≥
>>> 

Answer 2

unicodes need unicode format strings.

>>> print u'{0}'.format(s)
≥

Answer 3

A bit more information on why that happens.

>>> s = u'\u2265'
>>> print s

works because print automatically uses the system encoding for your environment, which was likely set to UTF-8. (You can check by doing import sys; print sys.stdout.encoding)

>>> print "{0}".format(s)

fails because format tries to match the encoding of the type that it is called on (I couldn't find documentation on this, but this is the behavior I've noticed). Since string literals are byte strings encoded as ASCII in python 2, format tries to encode s as ASCII, which then results in that exception. Observe:

>>> s = u'\u2265'
>>> s.encode('ascii')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2265' in position 0: ordinal not in range(128)

So that is basically why these approaches work:

>>> s = u'\u2265'
>>> print u'{}'.format(s)
≥
>>> print '{}'.format(s.encode('utf-8'))
≥

The source character set is defined by the encoding declaration; it is ASCII if no encoding declaration is given in the source file (https://docs.python.org/2/reference/lexical_analysis.html#string-literals)

Answer 4

Sorry to have to chime in 10 years later, but it looks like folks missed the simple answer here... str format strings need str arguments. Wherein '≥' is perfectly acceptable: since 2001 you can declare the source format. This lets you use UTF-8 directly in your source code:

# -*- coding: utf-8 -*-
print '{}'.format('≥')

Note that Python 3 drops format() and UTF-8 source support for byte strings, so consider switching to % and hex codes:

import sys
bprint = lambda s: sys.stdout.buffer.write(s + b"\n") and None
bprint(b'%s' % b'\xe2\x89\xa5')

(should you choose to port your program to Python 3 one day)