Problem
- We have a tool which produces output as json array
- We have another tool which anticipates input as ndjson (newline delimited json)
- I need to write a converter to transform json array to ndjson
Sample json array
[
{
"Property1": true,
"Property2": 0.1,
"Property3": "text"
},
{
"Property1": false,
"Property2": 0.2,
"Property3": "not text"
},
{
"Property1": true,
"Property2": 3.14,
"Property3": "sample"
},
{
"Property1": false,
"Property2": -51.0,
"Property3": "Property3"
}
]
Sample ndjson
{"Property1":true,"Property2":0.1,"Property3":"text"}
{"Property1":false,"Property2":0.2,"Property3":"not text"}
{"Property1":true,"Property2":3.14,"Property3":"sample"}
{"Property1":false,"Property2":-51.0,"Property3":"Property3"}
Design
- Keep the converter as simple as possible
- The converter should receive a source and a target file paths
- The converter should not parse the data just reformat it
- The converter should perform only some basic preliminary checks
Solution
- I'm comfortable to implement this with Json.NET
- But I wanted to practice with System.Text.Json
- So, I've decided to implement it with the latter
- I've added some comments to the code to help the reviewers
public static class JsonArrayToNDJsonConverter
{
static JsonWriterOptions writerOptions = new() { Indented = false };
public static void Convert(string sourcePath, string targetPath)
{
//Preliminary checks on source
if (!File.Exists(sourcePath))
throw new FileNotFoundException("Source file is not found");
using var sourceFile = new FileStream(sourcePath, FileMode.Open);
var jsonDocument = JsonDocument.Parse(sourceFile);
if (jsonDocument.RootElement.ValueKind != JsonValueKind.Array)
throw new InvalidOperationException("Json Array must be the source");
//Not that graceful handling of file existence
if (File.Exists(targetPath))
File.Delete(targetPath);
using var targetFile = new FileStream(targetPath, FileMode.CreateNew);
using var targetFileWriter = new StreamWriter(targetFile);
using var jsonObjectWithoutIndentation = new MemoryStream();
foreach (var jsonObject in jsonDocument.RootElement.EnumerateArray())
{
//Write json object without indentation into a memorystream
var jsonObjectStreamWriter = new Utf8JsonWriter(jsonObjectWithoutIndentation, writerOptions);
jsonObject.WriteTo(jsonObjectStreamWriter);
jsonObjectStreamWriter.Flush();
//Write memorystream to target
var singleLinedJson = Encoding.UTF8.GetString(jsonObjectWithoutIndentation.ToArray());
targetFileWriter.WriteLine(singleLinedJson);
//Reuse memory stream
jsonObjectWithoutIndentation.Position = 0;
jsonObjectWithoutIndentation.SetLength(0);
}
}
}