Is it possible to unmarshal json string representing a byte array within a struct?

Question

I have an API that returns a JSON object with a field containing a byte array (simplified):

{
    "value": "[208,188,149,77,179,245,29,184]"
}

I can unmarshal a string to a []byte just fine:

var test = make([]byte,0)

testData := []byte("[208,188,149,77,179,245,29,184]")

_ = json.Unmarshal(testData, &test)

fmt.Println(len(test)) // output: 8

However, when I try to unmarshal this via a struct, it doesn't:

type Test struct {
    Value []byte `json:"value"`
}

testData := []byte(`
    {
        "value": "[208,188,149,77,179,245,29,184]"
    }
`)

var test = Test{}
err := json.Unmarshal(testData, &test)
if err != nil {
    t.Fatal(err) // output: illegal base64 data at input byte 0
}

I get an error:

illegal base64 data at input byte 0

Is there anything I can do besides using json.RawMessage and manually unmarshalling this field separately?

Thanks.

That is a ridiculous format of api response. If arbitrary binary data needs to be encoded in json, a base 64 encoded string is the logical and standard way. — erik258
– erik258, Commented Aug 4, 2020 at 14:01
It's just a JSON string. If you want to treat the contents of that string as JSON, you'll need to unmarshal the value in that field after you unmarshal the outer struct. — Adrian
– Adrian, Commented Aug 4, 2020 at 14:01
@DanielFarrell: I completely agree, but sadly have no control over the API in use. — Jared
– Jared, Commented Aug 4, 2020 at 14:03
As @Adrian wrote: in your first example you unmarshal a JSON array, while in your 2nd example it's a JSON string, not an array. Change Test.Value to string, and do a second unmarshal, this time into a []byte. — icza
– icza, Commented Aug 4, 2020 at 14:04

Vaibhav Mishra · Accepted Answer · 2020-08-04 14:52:44Z

2

The reason why this isn't working is because the Unmarshaller is encountering an illegal value at the index 0, i.e., ". So this would work if your json string had the byte array unqouted because that's the correct syntax.

    {
        "value": [208,188,149,77,179,245,29,184]
    }

If you have no control over the response you get from the api one thing you can do is to write your own CustomMarshalJSON. You can find the code here. This will make your code cleaner and you wouldn't have to worry about unmarshalling the rawvalue every time you receive this kind of json value.

answered Aug 4, 2020 at 14:52

Vaibhav Mishra

4651 gold badge3 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Vaibhav Mishra Over a year ago

Can the folks downvoting the answer be considerate enough to mention what's wrong with it please? We are all trying to learn here.

Jared Over a year ago

This is exactly what I was looking for and is the cleanest approach I've seen. Don't worry about the downvotes. You helped me out with not only why my example wasn't working, a clean solution, and also an example. Thank you! And yes, still learning :)

Jared · Accepted Answer · 2020-08-04 14:16:21Z

As mentioned in the comments by @icza and @DanielFarrell, this does work:

type Test struct {
    RawValue string `json:"value"`
    Value []byte `json:"-"`
}

testData := []byte(`
    {
        "value": "[208,188,149,77,179,245,29,184]"
    }
`)

var test = Test{}
err := json.Unmarshal(testData, &test)
if err != nil {
    t.Fatal(err)
}

value := make([]byte, 0)
err = json.Unmarshal([]byte(test.RawValue), &value)
if err != nil {
    t.Fatal(err)
}

// assign the value to the original struct
test.Value = value

fmt.Println(len(test.Value)) // output: 8

Collectives™ on Stack Overflow

Is it possible to unmarshal json string representing a byte array within a struct?

2 Answers 2

2 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Related