Loading JSON object in Python using urllib.request and json modules

Question

I am having problems making the modules 'json' and 'urllib.request' work together in a simple Python script test. Using Python 3.5 and here is the code:

import json
import urllib.request

urlData = "http://api.openweathermap.org/data/2.5/weather?q=Boras,SE"
webURL = urllib.request.urlopen(urlData)
print(webURL.read())
JSON_object = json.loads(webURL.read()) #this is the line that doesn't work

When running script through command line the error I am getting is "TypeError:the JSON object must be str, not 'bytes'". I am new to Python so there is most likely a very easy solution to is. Appreciate any help here.

Martijn Pieters · Accepted Answer · 2017-09-12 15:33:28Z

Apart from forgetting to decode, you can only read the response once. Having called .read() already, the second call returns an empty string.

Call .read() just once, and decode the data to a string:

data = webURL.read()
print(data)
encoding = webURL.info().get_content_charset('utf-8')
JSON_object = json.loads(data.decode(encoding))

The response.info().get_content_charset() call tells you what characterset the server thinks is used.

Demo:

>>> import json
>>> import urllib.request
>>> urlData = "http://api.openweathermap.org/data/2.5/weather?q=Boras,SE"
>>> webURL = urllib.request.urlopen(urlData)
>>> data = webURL.read()
>>> encoding = webURL.info().get_content_charset('utf-8')
>>> json.loads(data.decode(encoding))
{'coord': {'lat': 57.72, 'lon': 12.94}, 'visibility': 10000, 'name': 'Boras', 'main': {'pressure': 1021, 'humidity': 71, 'temp_min': 285.15, 'temp': 286.39, 'temp_max': 288.15}, 'id': 2720501, 'weather': [{'id': 802, 'description': 'scattered clouds', 'icon': '03d', 'main': 'Clouds'}], 'wind': {'speed': 5.1, 'deg': 260}, 'sys': {'type': 1, 'country': 'SE', 'sunrise': 1443243685, 'id': 5384, 'message': 0.0132, 'sunset': 1443286590}, 'dt': 1443257400, 'cod': 200, 'base': 'stations', 'clouds': {'all': 40}}

Jaykumar Patel · Accepted Answer · 2018-06-14 07:51:17Z

As I study myself you just need to use decode('utf-8') function, then after use json.load() function to extract into json format.

>>> import json
>>> import urllib.request

>>> urlData = "http://api.openweathermap.org/data/2.5/weather?q=Boras,SE"
>>> webURL = urllib.request.urlopen(urlData)
>>> data = webURL.read()
>>> JSON_object = json.loads(data.decode('utf-8'))
{'coord': {'lat': 57.72, 'lon': 12.94}, 'visibility': 10000, 'name': 'Boras', 'main': {'pressure': 1021, 'humidity': 71, 'temp_min': 285.15, 'temp': 286.39, 'temp_max': 288.15}, 'id': 2720501, 'weather': [{'id': 802, 'description': 'scattered clouds', 'icon': '03d', 'main': 'Clouds'}], 'wind': {'speed': 5.1, 'deg': 260}, 'sys': {'type': 1, 'country': 'SE', 'sunrise': 1443243685, 'id': 5384, 'message': 0.0132, 'sunset': 1443286590}, 'dt': 1443257400, 'cod': 200, 'base': 'stations', 'clouds': {'all': 40}}

Max Kleiner · Accepted Answer · 2021-08-02 11:37:46Z

1

with an evalstr:

   println('openurl: '+ 
   eng.EvalStr('json.loads(urllib.request.urlopen("'+
                           urlData+'").read().decode("utf-8"))'));

answered Aug 2, 2021 at 11:37

Max Kleiner

1,6601 gold badge15 silver badges14 bronze badges

Collectives™ on Stack Overflow

Loading JSON object in Python using urllib.request and json modules

3 Answers 3

Comments

Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Linked

Related