How to get the json data from the url using Python script apart from requests library?

Question

I have a URL which gives me the below JSON String if I hit them on the browser -

Below is my URL, let's say it is URL-A and I have around three URL's -

http://hostnameA:1234/Service/statistics?%24format=json

And below is my JSON String -

{
 "description": "",
 "statistics": {
  "dataCount": 0,
 }
}

Now I have written a Python script which is scanning all my 3 URL's and then parse then JSON String to extract the value of dataCount from it. And it should keep on running every few seconds to scan the URL and then parse it.

Below are my URL's

hostnameA       http://hostnameA:1234/Service/statistics?%24format=json
hostnameB       http://hostnameB:1234/Service/statistics?%24format=json
hostnameC       http://hostnameC:1234/Service/statistics?%24format=json

And the data I want to see is like this on the console, here dataCount will be actual number

hostnameA - dataCount
hostnameB - dataCount
hostnameC - dataCount

And I have the below python script which works fine locally on my cygwin but if I am running it on my company's production ubuntu machine it gives an error -

import requests
from time import sleep

def get_data_count(url):
    try:
        req = requests.get(url)
    except requests.ConnectionError:
        return 'could not get page'

    try:
        # this line is giving an error
        return int(req.json['statistics']['dataCount'])
    except TypeError:
        return 'field not found'
    except ValueError:
        return 'not an integer'

def main():
    urls = [
        ('hostnameA', 'http://hostnameA:1234/Service/statistics?%24format=json'),
        ('hostnameB', 'http://hostnameB:1234/Service/statistics?%24format=json'),
        ('hostnameC', 'http://hostnameC:1234/Service/statistics?%24format=json')
    ]

    while True:
        print('')
        for name, url in urls:
            res = get_data_count(url)
            print('{name} - {res}'.format(name=name, res=res))
        sleep(10.)

if __name__=="__main__":
    main()

Below is the error I am getting -

AttributeError: 'Response' object has no attribute 'json'

I am using Python 2.7.3 and Ubuntu 12.04 and the version of requests I am running is 0.8.2 (I guess this is the problem).

In any case, is there any way I can rewrite the above script using some other library apart from requests meaning just the portion of getting the data from server, we can use other libraries right?

Since I guess, I cannot update this package since it's our production ubuntu servers so I need to find some other way to do this.

Indeed, requests 0.8.2 is ancient and does not support .json. You can install a newer version in a virtualenv, which will then be used instead. — Martijn Pieters
– Martijn Pieters, Commented Apr 26, 2014 at 16:56
@MartijnPieters: Thanks, but how do I install virutualenv? I have never use it before. If you can guide me then it will be of great help. Not a python expert :( — AKIWEB
– AKIWEB, Commented Apr 26, 2014 at 16:59
There is a Ubuntu package; that's for Precise Pangolin, the Ubuntu version of your server (judging by the 0.8.2 version tag of requests). — Martijn Pieters
– Martijn Pieters, Commented Apr 26, 2014 at 17:02

Martijn Pieters · Accepted Answer · 2014-04-26 17:04:21Z

You can use requests still, just not rely on the response object to do the decoding for you:

import json

# ...
data = json.loads(req.content)
return int(data['statistics']['dataCount'])

Version 0.8.2 is ancient; you can use virtualenv to create a location for you to install a newer version instead:

$ virtualenv venv
New python executable in venv/bin/python2.7
Also creating executable in venv/bin/python
Installing Setuptools..............................................................................................................................................................................................................................done.
Installing Pip.....................................................................................................................................................................................................................................................................................................................................done.
$ cd venv/
$ bin/pip install requests
Downloading/unpacking requests
  Downloading requests-2.2.1.tar.gz (421kB): 421kB downloaded
  Running setup.py egg_info for package requests

Installing collected packages: requests
  Running setup.py install for requests

Successfully installed requests
Cleaning up...
$ bin/python -c 'import requests; print requests.__version__'
2.2.1

Collectives™ on Stack Overflow

How to get the json data from the url using Python script apart from requests library?

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related