Unmarshall json value into []byte where the string could sometimes be escaped json

Question

I have a json response that looks like this

{
  "eventId":"fbf4a1a1-b4a3-4dfe-a01f-ec52c34e16e4",
  "eventType":"event-type",
  "eventNumber":0,
  "data":"{\n  \"a\": \"1\"\n}",
  "metaData":"{\n  \"yes\": \"no\"\n}",
  "streamId":"test",
  "isJson":true,
  "isMetaData":true,
  "isLinkMetaData":false,
  "positionEventNumber":0,
  "positionStreamId":"test",
  "title":"0@test",
  "id":"http://localhost:2113/streams/test/0",
  "updated":"2017-12-14T05:09:58.816079Z"
}

the key value pairs of data, and metaData might sometimes be encoded json or it might not.

I want to decode those values into a byte array like this.

// Event represent an event to be stored.
type Event struct {
    Data      []byte    `json:"data"`
    Metadata  []byte    `json:"metaData"`
}

but when I try to unmarshal the json object I get the following error:

illegal base64 data at input byte 0

What could I be doing wrong here?

It works fine if I decode the data and metaData into a string, but I don't want to use a string.

From the docs: []byte encodes as a base64-encoded string, and the error also indicates that it's looking for base64 data. Can you just unmarshal into a string, and convert it as necessary? — Mr_Pink
– Mr_Pink, Commented Dec 14, 2017 at 21:18
@JimB I could, but isn't there another way for []byte not to decode that way? I don't want to have to be converting from []byte to string all the time. — Victor Martinez
– Victor Martinez, Commented Dec 14, 2017 at 21:21

David Budworth · Accepted Answer · 2017-12-14 22:29:44Z

You are looking for the json.RawMessage type

It is just a specialized []byte that you can then use as need be.

type Event struct {
    Data      json.RawMessage    `json:"data"`
    Metadata  json.RawMessage    `json:"metaData"`
}

Then you could treat it as a literal []byte via []byte(e.Data)

Here's an example of use, on play:

package main

import (
    "encoding/json"
    "fmt"
)

var RAW = []byte(`
{
  "eventId":"fbf4a1a1-b4a3-4dfe-a01f-ec52c34e16e4",
  "eventType":"event-type",
  "eventNumber":0,
  "data":"{\n  \"a\": \"1\"\n}",
  "metaData":"{\n  \"yes\": \"no\"\n}",
  "streamId":"test",
  "isJson":true,
  "isMetaData":true,
  "isLinkMetaData":false,
  "positionEventNumber":0,
  "positionStreamId":"test",
  "title":"0@test",
  "id":"http://localhost:2113/streams/test/0",
  "updated":"2017-12-14T05:09:58.816079Z"
}
`)

type Event struct {
    Data     json.RawMessage `json:"data"`
    Metadata json.RawMessage `json:"metaData"`
}

func main() {
    var e Event
    err := json.Unmarshal(RAW, &e)
    fmt.Printf("%v -- %+v\n", err, e)
    b, err := json.Marshal(e)
    fmt.Printf("%v -- %s\n", err, b)
}

sorry, I meant that it fails my tests, because I'm expecting that it will be nil when it unmarshalls, but instead I get a byte array representing nil.

Victor Martinez · Accepted Answer · 2017-12-14 23:26:11Z

I created a type that implements the TextUnmarshaler and TextMarshaler interfaces. The json decoder looks for this if the type doesn't implement MarshalJSON and UnmarshalJSON methods.

type RawData []byte

func (r RawData) MarshalText() (text []byte, err error) {
    return r[:], nil
}

func (r *RawData) UnmarshalText(text []byte) error {
    *r = text[:]

    return nil
}

// Event represent an event to be stored.
type Event struct {
    Data      RawData   `json:"data,omitempty"`
    Metadata  RawData   `json:"metaData,omitempty"`

}

I needed this because sometimes the Data or Metadata would not be json encoded in a string, but could also be other formats like protocol buffers.

Collectives™ on Stack Overflow

Unmarshall json value into []byte where the string could sometimes be escaped json

2 Answers 2

3 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Related