10

I have a JSON string that I expect to contain duplicate keys that I am unable to make JSON.NET happy with.

I was wondering if anybody knows the best way (maybe using JsonConverter? ) to get JSON.NET to change a JObject's child JObjects into to JArrays when it sees duplicate key names ?

// For example: This gives me a JObject with a single "JProperty\JObject" child.
var obj = JsonConvert.DeserializeObject<object>("{ \"HiThere\":1}");

// This throws:
// System.ArgumentException : Can not add Newtonsoft.Json.Linq.JValue to Newtonsoft.Json.Linq.JObject.
obj = JsonConvert.DeserializeObject<object>("{ \"HiThere\":1, \"HiThere\":2, \"HiThere\":3 }");

The actual JSON I am trying to deserialize is much more complicated and the duplicates are nested at multiple levels. But the code above demonstrates why it fails for me.

I understand that the JSON is not recommended which is why I am asking if JSON.NET has a way to work around this. For argument's sake let's say I do not have control over the JSON. I actually do use a specific type for the parent object but the particular property that is having trouble will either be a string or another nested JSON object. The failing property type is "object" for this reason.

1
  • 1
    Actually, JSON objects with duplicate names are perfectly legal. ECMA-404/json.org are silent on the issue. Meanwhile RFC7159 Section 4 says The names within an object SHOULD be unique (SHOULD = recommended, not MUST) along with some discussion of the implications. Commented Jun 17, 2016 at 17:18

3 Answers 3

8

While a JObject cannot contain properties with duplicate names, the JsonTextReader used to populate it during deserialization does not have such a restriction (this makes sense if you think about it: it's a forward-only reader; it is not concerned with what it has read in the past).

Here is some code that will populate a hierarchy of JTokens, converting property values to JArrays as necessary if a duplicate property name is encountered in a particular JObject.

Since I don't know your actual JSON and requirements, you may need to make some adjustments to it, but it's something to start with at least.

Here's the code:

public static JToken DeserializeAndCombineDuplicates(JsonTextReader reader)
{
    if (reader.TokenType == JsonToken.None)
    {
        reader.Read();
    }

    if (reader.TokenType == JsonToken.StartObject)
    {
        reader.Read();
        JObject obj = new JObject();
        while (reader.TokenType != JsonToken.EndObject)
        {
            string propName = (string)reader.Value;
            reader.Read();
            JToken newValue = DeserializeAndCombineDuplicates(reader);

            JToken existingValue = obj[propName];
            if (existingValue == null)
            {
                obj.Add(new JProperty(propName, newValue));
            }
            else if (existingValue.Type == JTokenType.Array)
            {
                CombineWithArray((JArray)existingValue, newValue);
            }
            else // Convert existing non-array property value to an array
            {
                JProperty prop = (JProperty)existingValue.Parent;
                JArray array = new JArray();
                prop.Value = array;
                array.Add(existingValue);
                CombineWithArray(array, newValue);
            }

            reader.Read();
        }
        return obj;
    }

    if (reader.TokenType == JsonToken.StartArray)
    {
        reader.Read();
        JArray array = new JArray();
        while (reader.TokenType != JsonToken.EndArray)
        {
            array.Add(DeserializeAndCombineDuplicates(reader));
            reader.Read();
        }
        return array;
    }

    return new JValue(reader.Value);
}

private static void CombineWithArray(JArray array, JToken value)
{
    if (value.Type == JTokenType.Array)
    {
        foreach (JToken child in value.Children())
            array.Add(child);
    }
    else
    {
        array.Add(value);
    }
}

And here's a demo:

class Program
{
    static void Main(string[] args)
    {
        string json = @"
        {
            ""Foo"" : 1,
            ""Foo"" : [2],
            ""Foo"" : [3, 4],
            ""Bar"" : { ""X"" : [ ""A"", ""B"" ] },
            ""Bar"" : { ""X"" : ""C"", ""X"" : ""D"" },
        }";

        using (StringReader sr = new StringReader(json))
        using (JsonTextReader reader = new JsonTextReader(sr))
        {
            JToken token = DeserializeAndCombineDuplicates(reader);
            Dump(token, "");
        }
    }

    private static void Dump(JToken token, string indent)
    {
        Console.Write(indent);
        if (token == null)
        {
            Console.WriteLine("null");
            return;
        }
        Console.Write(token.Type);

        if (token is JProperty)
            Console.Write(" (name=" + ((JProperty)token).Name + ")");
        else if (token is JValue)
            Console.Write(" (value=" + token.ToString() + ")");

        Console.WriteLine();

        if (token.HasValues)
            foreach (JToken child in token.Children())
                Dump(child, indent + "  ");
    }
}

Output:

Object
  Property (name=Foo)
    Array
      Integer (value=1)
      Integer (value=2)
      Integer (value=3)
      Integer (value=4)
  Property (name=Bar)
    Array
      Object
        Property (name=X)
          Array
            String (value=A)
            String (value=B)
      Object
        Property (name=X)
          Array
            String (value=C)
            String (value=D)
Sign up to request clarification or add additional context in comments.

1 Comment

Thanks Brian Rogers - You've filled in some gaps in my understanding of the JsonTextReader. I actually have a JsonConverter already assigned to my "object" property that I was using to abstract away the traces of JSON.NET. But in my unit test I was just trying to deserialize my "bad" JSON directly without the converter so it didn't occur to me that I could edit that to resolve the issue. I will post the code below as an additional answer.
3

Brian Rogers - Here is the helper function of the JsonConverter that I wrote. I modified it based on your comments about how a JsonTextReader is just a forward-reader doesn't care about duplicate values.

private static object GetObject(JsonReader reader)
{
    switch (reader.TokenType)
    {
        case JsonToken.StartObject:
        {
            var dictionary = new Dictionary<string, object>();

            while (reader.Read() && (reader.TokenType != JsonToken.EndObject))
            {
                if (reader.TokenType != JsonToken.PropertyName)
                    throw new InvalidOperationException("Unknown JObject conversion state");

                string propertyName = (string) reader.Value;

                reader.Read();
                object propertyValue = GetObject(reader);

                object existingValue;
                if (dictionary.TryGetValue(propertyName, out existingValue))
                {
                    if (existingValue is List<object>)
                    {
                        var list = existingValue as List<object>;
                        list.Add(propertyValue);
                    }
                    else
                    {
                        var list = new List<object> {existingValue, propertyValue};
                        dictionary[propertyName] = list;
                    }
                }
                else
                {
                    dictionary.Add(propertyName, propertyValue);
                }
            }

            return dictionary;
        }
        case JsonToken.StartArray:
        {
            var list = new List<object>();

            while (reader.Read() && (reader.TokenType != JsonToken.EndArray))
            {
                object propertyValue = GetObject(reader);
                list.Add(propertyValue);
            }

            return list;
        }
        default:
        {
            return reader.Value;
        }
    }
}

Comments

-1

You should not be using a generic type of object, it should be a more specific type.

However you json is malformed which is you rmain problem

You have :

"{ \"HiThere\":1, \"HiThere\":2, \"HiThere\":3 }"

But it should be:

"{"HiTheres": [{\"HiThere\":1}, {\"HiThere\":2}, {\"HiThere\":3} ]}"

Or

"{ \"HiThereOne\":1, \"HiThereTwo\":2, \"HiThereThree\":3 }"

You json is one object with 3 fields with all the same name ("HiThere"). Which wont work.

The json I have shown gives: An array (HiTheres) of three objects each with a field of HiThere Or One object with three field with different names. (HiThereOne, HiThereTwo, "HiThereThree)

Have a look at http://jsoneditoronline.org/index.html And http://json.org/

3 Comments

Did you use a proper type? e.g., JsonConvert.DeserializeObject<HiThereClass>()
Although you're right about the duplicate names, "{"HiTheres": [\"HiThere\":1, \"HiThere\":2, \"HiThere\":3 ]}" is still not valid JSON.
Copy paste fail, try now.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.