Getting KeyError when parsing JSON in Python for following response

Question

TL;DR: Confused on how to parse following JSON response and get the value of [status of 12345 of dynamicValue_GGG of payload] in this case.

Full question: I get the following as (sanitized) response upon hitting a REST API via Python code below:

response = requests.request("POST", url, data=payload, headers=headers).json()

{
    "payload": {
        "name": "asdasdasdasd",
        "dynamicValue_GGG": {
            "12345": {
                "model": "asad",
                "status": "active",
                "subModel1": {
                    "dynamicValue_67890": {
                        "model": "qwerty",
                        "status": "active"
                    },
                "subModel2": {
                    "dynamicValue_33445": {
                        "model": "gghjjj",
                        "status": "active"
                    },
                "subModel3": {
                    "dynamicValue_66778": {
                        "model": "tyutyu",
                        "status": "active"
                    }                   
                }
            }
        },
        "date": "2016-02-04"
    },
    "design": "asdasdWWWsaasdasQ"
}

If I do a type(response['payload']), it gives me 'dict'.

Now, I'm trying to parse the response above and fetch certain keys and values out of it. The problem is that I'm not able to iterate through using "index" and rather have to specify the "key", but then the response has certain "keys" that are dynamically generated and sent over. For instance, the keys called "dynamicValue_GGG", "dynamicValue_66778" etc are not static unlike the "status" key.

I can successfully parse by mentioning like:

print response['payload']['dynamicValue_GGG']['12345'][status]

in which case I get the expected output = 'active'.

However, since I have no control on 'dynamicValue_GGG', it would work only if I can specify something like this instead:

print response['payload'][0][0][status]

But the above line gives me error: " KeyError: 0 " when the python code is executed.

Is there someway in which I can use the power of both keys as well as index together in this case?

Do you know anything about the dynamic value? Does it always start with "dynamicValue_"? — Barmar
– Barmar, Commented Feb 4, 2017 at 7:42
Dictionaries don't have any inherent order, you can't use a numeric index to access its elements. — Barmar
– Barmar, Commented Feb 4, 2017 at 7:44
They must be something you can look up from some other part of the API, like a company ID, transaction ID, etc. I find it hard to believe that the API would just insert random keys in the response and expect any client application to be able to parse them. — Barmar
– Barmar, Commented Feb 4, 2017 at 7:51
I suspect you haven't shown the JSON accurately. In the 12345 object, you have three subModel: properties, but keys have to be unique. There's probably just a single subModels: property, whose value is an array. Maybe all the dynamic values are in a single object, that makes it easier to parse. — Barmar
– Barmar, Commented Feb 4, 2017 at 7:55
There has to be some other API you can call to get the dynamic value. Otherwise there's just no reasonable way to process this response. — Barmar
– Barmar, Commented Feb 4, 2017 at 8:18

Ferry Boender · Accepted Answer · 2017-02-04 08:06:10Z

2

The order of values in a dictionary in Python are random, so you cannot use indexing. You'll have to iterate over all elements, potentially recursive, and test to see if it's the thing you're looking for. For example:

def find_submodels(your_dict):
  for item_key, item_values in your_dict.items():
    if 'status' in item_values:
      print item_key, item_values['status']

    if type(item_values) == dict:
      find_submodels(item_values)

find_submodels(your_dict)

Which would output:

12345 active
dynamicValue_67890 active
dynamicValue_66778 active
dynamicValue_33445 active

answered Feb 4, 2017 at 8:06

Ferry Boender

7881 gold badge8 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Barmar Over a year ago

status is 2 levels down from the dynamic value. It should be if 'status' in item_values['12345']:

Barmar Over a year ago

Also, you should test whether item_values is a dict before you use if 'status' in item_values:. You don't want to do that on the name and date items, since their values are strings.

Ferry Boender Over a year ago

This code is obviously not fool proof. It's merely a demonstration of how to iterate through dynamic hierarchical data. The OP's posted data isn't even valid Python, so it's probably going to be different in his real problem.

Barmar Over a year ago

It's not supposed to be valid Python, it's JSON.

maverick Over a year ago

Thanks Barmar and Ferry Boender for looking into this. I followed Ferry's suggested code and was able to parse further through the dynamic code. Had to use if elif else to parse down the tree of dynamic values, but the idea was same. Thanks again!

|

Collectives™ on Stack Overflow

Getting KeyError when parsing JSON in Python for following response

1 Answer 1

6 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Related