Python JSON loads - wrong encoding of file

Question

in Python 3.8.2 I trying to open the JSON file and read data from it. JSON file is encoded in UTF-8 but when I read it, final string seems as to by wrongly opened as ANIS.

Tried with Notepad++:

JSON file opened in UTF8:

"úterý, třicátého-prvního, sedumnáct v jedna dvacet dva"

Changed coding to ANIS:

"ĂşterĂ˝, tĹ™icĂˇtĂ©ho-prvnĂho, sedumnĂˇct v jedna dvacet dva"

When I copy string directly to python file, it is read properly.

My program:

  import json  

print('Wrong characters:')
with open('C:/Users/Tony/Desktop/test.json',
                          'r') as jfile:
                        data = json.loads(
                        jfile.read())
print(data)

print('Ok Characters:')
data2 = json.loads('{ "td": { "1": { "as": "úterý, třicátého-prvního, sedumnáct v jedna dvacet dva"}, "2": { "as": "úterý, třicátého-prvního, sedumnáct v třináct dvacet dva"}}}')
print(data2)

print('System settings:')
import sys; print(sys.stdout)

Console output:

Wrong characters:
{'td': {'1': {'as': 'ĂşterĂ˝, tĹ™icĂˇtĂ©ho-prvnĂ\xadho, sedumnĂˇct v jedna dvacet dva'}, '2': {'as': 'ĂşterĂ˝, tĹ™icĂˇtĂ©ho-prvnĂ\xadho, sedumnĂˇct v tĹ™inĂˇct dvacet dva'}}}
Ok Characters:
{'td': {'1': {'as': 'úterý, třicátého-prvního, sedumnáct v jedna dvacet dva'}, '2': {'as': 'úterý, třicátého-prvního, sedumnáct v třináct dvacet dva'}}}
System settings:
<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>

test.json file

Raphael Medaer · Accepted Answer · 2020-04-10 10:36:41Z

3

It's difficult to debug without the preferred encoding of your system. You can get it with:

import locale
locale.getpreferredencoding(False)

But I suspect that your encoding is not ASCII or UTF8. Since Python3, open takes an argument encoding. You should try to specify the encoding utf-8.

import json

with open('example.json', 'r', encoding='utf-8') as f:
    print(json.loads(f.read()))

edited Apr 10, 2020 at 10:36

answered Apr 9, 2020 at 23:17

Raphael Medaer

2,53814 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Tony762 Over a year ago

Nice one. print(locale.getpreferredencoding(False)) show that system locale is cp1250. Setting encoding='UTF8' with open do the trick. Firstly I tried to change it with json.loads() but setting it there was deprecated.

Collectives™ on Stack Overflow

Python JSON loads - wrong encoding of file

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related