5

I have a ndjson (newline delimited JSON) file, I need to parse it and get the data for some logical operation. Is there any good method for parsing ndjson files using golang. A sample ndjson is given below

{"a":"1","b":"2","c":[{"d":"100","e":"10"}]}
{"a":"2","b":"2","c":[{"d":"101","e":"11"}]}
{"a":"3","b":"2","c":[{"d":"102","e":"12"}]}
1

1 Answer 1

8

The encoding/json Decoder parses sequential JSON documents with optional or required whitespace depending on the value type. Because newlines are whitespace, the decoder handles ndjson.

d := json.NewDecoder(strings.NewReader(stream))
for {
    // Decode one JSON document.
    var v interface{}
    err := d.Decode(&v)

    if err != nil {
        // io.EOF is expected at end of stream.
        if err != io.EOF {
            log.Fatal(err)
        }
        break
    }

    // Do something with the value.
    fmt.Println(v)
}

Run it on the playground.

Sign up to request clarification or add additional context in comments.

2 Comments

FTR: Decoder doesn't require any kind of separation between documents: play.golang.org/p/oHc5xaegQuC.
@Peter I updated answer regarding whitespace. Separation is required for some value types: play.golang.org/p/ztaeA5UYKlo

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.