Marshaling JSON []byte as strings in Go

Question

I have a struct containing strings as []byte fields which I'd like to encode into JSON. However, the generated JSON contains a non expected string representation of the slice contents. Here is an example of what I refer:

package main

import (
    "fmt"
    "encoding/json"
    )

type Msg struct {
    Content []byte
}

func main() {
    helloStr := "Hello"
    helloSlc := []byte(helloStr)
    fmt.Println(helloStr, helloSlc)

    obj := Msg{helloSlc}
    json, _ := json.Marshal(obj)
    fmt.Println(string(json))
}

This produces the following output:

Hello [72 101 108 108 111]
{"Content":"SGVsbG8="}

What kind of conversion is the json.Marshal() method performing to the []byte encoded string. How can I generate a JSON with the original content of my string {"Content":"Hello"}?

Community · Accepted Answer · 2017-05-23 12:01:05Z

20

A []byte is marshalled as base64 encoded string. From the documentation:

Array and slice values encode as JSON arrays, except that []byte encodes as a base64-encoded string, and a nil slice encodes as the null JSON object.

These values are decoded properly when unmarshalled.

The reason why this is done is that JSON does not have a native representation for raw bytes. See this question for a detailed explanation.

edited May 23, 2017 at 12:01

CommunityBot

11 silver badge

answered Apr 7, 2016 at 1:30

nemo

58.3k15 gold badges144 silver badges145 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

prl900 Over a year ago

So, I guess the only way of properly formatting my JSON would be to cast the []byte into a string before encoding it. Or is there a better approach to this problem?

nemo Over a year ago

Your JSON is properly formatted. This behaviour is not a bug and intended. If you want a string representation and are dealing with text only, use string instead of []byte.

Zhiwei Over a year ago

Just in case anyone wondering how to cast []byte to string, you can do mystr := string(someByteArray) in Go to convert the byte array to a string.

silverfighter · Accepted Answer · 2018-10-01 11:25:08Z

5

I came a cross the same thing and even if this is a rather old question and already answered there is another option.

If you use json.RawMessage (which internaly is a []byte) as a type instead of []byte the Marshaling works into a Json String as expected.

answered Oct 1, 2018 at 11:25

silverfighter

6,89212 gold badges52 silver badges76 bronze badges

1 Comment

tutuDajuju Over a year ago

Do note that the resulting byte slice will differ from the JSON value because it will be wrapped in double quotes, i.e it contains 2 extra " symbols on each end of the slice

boy9527 · Accepted Answer · 2023-06-05 01:17:20Z

You can use a third party library to achieve what you want

package main

import (
    "fmt"
    "strconv"
    "strings"
    "time"
    "unsafe"

    jsoniter "github.com/json-iterator/go"
)

func main() {
    err := test()
    if err != nil {
        panic(err)
    }
}

func test() error {
    jsoniter.RegisterTypeEncoder("[]uint8", &uint8Enc{})

    var a struct {
        A []byte `json:"a"`
        B []byte `json:"b"`
    }
    a.A = []byte{'1', 1, 2, 3, '9'}
    a.B = []byte(time.Now().String())

    s, err := json.MarshalIndent(a, "", "  ")
    if err != nil {
        return err
    }
    fmt.Println(string(s))
    return nil
}

var json = jsoniter.ConfigCompatibleWithStandardLibrary

type uint8Enc struct{}

func (ue *uint8Enc) IsEmpty(ptr unsafe.Pointer) bool {
    data := *((*[]uint8)(ptr))
    return len(data) == 0
}
func (ue *uint8Enc) Encode(ptr unsafe.Pointer, stream *jsoniter.Stream) {
    var (
        data = *((*[]uint8)(ptr))
        sb   strings.Builder
    )
    const hexTable = "0123456789abcdef"
    for _, v := range data {
        if strconv.IsPrint(rune(v)) {
            sb.WriteByte(v)
        } else {
            sb.WriteString(`\x`)
            sb.WriteByte(hexTable[v>>4])
            sb.WriteByte(hexTable[v&0x0f])
        }
    }
    stream.WriteString(sb.String())
}

The result is as follows

{
  "a": "1\\x01\\x02\\x039",
  "b": "2023-06-05 09:15:38.551518 +0800 CST m=+0.003193401"
}

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review

Collectives™ on Stack Overflow

Marshaling JSON []byte as strings in Go

3 Answers 3

3 Comments

1 Comment

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

1 Comment

2 Comments

Linked

Related