6

My Question is similar to what has been asked here . few points :

  1. I can not change the format. (No commas to be added etc)
  2. This is basically a huge .txt file containing 1000's of Json objects.
  3. My Json objects are HUGE.

This is what I am doing right now : 

    FileReader fileReader = new FileReader(fileName);
        BufferedReader reader = new BufferedReader(fileReader);
        String data = "";
        while((data = reader.readLine()) != null){
            ObjectMapper mapper = new ObjectMapper();
            Map<String,String> map = mapper.readValue(data, Map.class);
        }

Currently I am using Jackson and Ideally I would like to read one Json Object from the file at a time, Parse it and then move on to the next one. I need to count let say unique number of id's from these Json object and do more operations. It will be best to read them one by one.

Is jackson would be the best way going forward ? This is a good example of parsing huge Json, But it deals with only one object per file. My file has huge Jsons (1000s of them).

Improve this question
8
  • Q: Have you tried parsing JSON with the Jackson Streaming API (per your link)? Are you sure it can't handle multiple JSON objects in a single file? Commented Aug 7, 2015 at 6:14
  • I have tried doing exactly same thing being done in This link. It parses only first json object and stops. Commented Aug 7, 2015 at 6:44
  • 1
    Does the parser takes a InputStream or Reader as a parameter? In that case it might be possible to loop reading objects till the InputStream reaches EOF (check for available). Commented Aug 7, 2015 at 7:01
  • That might work. It does have a reader. It is iterating through different objects now, but its messed up for nested objects within. I will try to fix it and see if it works. Thanks. Commented Aug 7, 2015 at 7:45
  • If you want to use Jackson why did you tag this question Gson? Commented Aug 7, 2015 at 14:35

3 Answers 3

Reset to default
7

Here is a Jackson example that works for me. I have thousands json objects (tokens) in a single json file. This code will iterate through the file read each token and print it's serial.

Required imports: 

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.ObjectMapper;

Using Jackson to read multiple json objects from FileInputStream:

try (FileInputStream fis = new FileInputStream("D:/temp/tokens.json")) {
        JsonFactory jf = new JsonFactory();
        JsonParser jp = jf.createParser(fis);
        jp.setCodec(new ObjectMapper());
        jp.nextToken();
        while (jp.hasCurrentToken()) {
            Token token = jp.readValueAs(Token.class);
            jp.nextToken();
            System.out.println("Token serial "+token.getSerialNumber());
        }
    }
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

4

Here is a more JAVA 8ish solution for your query, I always lean toward BufferedReader over InputStreams for any place where parsing is going to be done a lot of time. 

 ObjectMapper mapper  = new ObjectMapper();
 JsonFactory jsonFactory = new JsonFactory();
 try(BufferedReader br = new BufferedReader(new FileReader("luser.txt"))) {
     Iterator<luser> value = mapper.readValues( jsonFactory.createParser(br), luser.class);
     value.forEachRemaining((u)->{System.out.println(u);});
 }

The deserialization for each object happens as part of next(), in each iteration.

Improve this answer

Comments

3

Here is how I used Gson's JSONReader API to handle similar requirement as above 

public static List<YOURPOJO> readTraceLog(String filepath) throws IOException {

         Gson gson = new Gson();
         JsonReader jsonReader = new JsonReader(new FileReader(filepath));
         // important as handles unwanted formatting stuffs such empty spaces 
        jsonReader.setLenient(true);

        boolean start = true; // start of read
        jsonReader.beginObject(); // first object begins
        //List to hold object 
        List<YOURPOJO> completeList = new ArrayList<YOURPOJO>();
        //YOURPOJO has two attributes one is ID and other is list of ANOTHERPOJO

        while (jsonReader.hasNext()) {
            if (!start) {
                //to stop end of Document
                if (jsonReader.peek().toString().matches("END_DOCUMENT")) {
                    break;
                }
                //continue reading object as the come -{
                jsonReader.beginObject();

            }
            start = false;
            YOURPOJO pojo = new YOURPOJO();
            //read attribute id
            String name = jsonReader.nextName();
            pojo.setId(name);

            //list to store ANOTHERPOJO objects
            List<ANOTHERPOJO> tempList = new ArrayList<ANOTHERPOJO>();

            //begin reading list - [
            jsonReader.beginArray();
            while (jsonReader.hasNext()) {
                ANOTHERPOJO t = gson.fromJson(jsonReader, ANOTHERPOJO.class);
                tempList.add(t);
            }
            //end reading list - ]
            jsonReader.endArray();

            //store data
            pojo.setTraceDetails(tempList);
            completeList.add(YOURPOJO);

            //end of object  - }
            jsonReader.endObject();

        }

        jsonReader.close();
        return completeList;
    }
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.