How do I parse a large XML file using Java?

Question

I am trying to parse an XML file using Java.

The XML file size is 256 kb only. I am using a DOM parser to parse the XML file. How can I parse the large XML file content?

Here's the method that parses the file content:

public Document parse_a_string(StringBuffer decodedFile) {
    Document doc1 = null;
    try {
        DocumentBuilderFactory factory =
                DocumentBuilderFactory.newInstance();
        DocumentBuilder db = factory.newDocumentBuilder();
        InputSource inStream = new InputSource();

         // problem here
        inStream.setCharacterStream(new StringReader(decodedFile.toString()));

        doc1 = db.parse(inStream);
    } catch (Exception e) {
    }
    return doc1;
}

The file content is in the StringBuffer reference object, decodedFile, but when I set it to StringReader it accept only string.

StringBuffer has a toString() method to convert it to String. Check in JavaDoc before posting these questions — Aravind Yarram
– Aravind Yarram, Commented Feb 8, 2012 at 16:35
yes, but DecodedFile size is larger than string size. so we need to use stringbuffer. but in StringReader accept only string objects. my problem is how can we do parse large content which is present in stringbuffer. — Mohan
– Mohan, Commented Feb 8, 2012 at 16:41
Are you getting any errors while parsing it, like OutOfMemory? It might be something as simple as changing the Java memory settings. — Spencer Kormos
– Spencer Kormos, Commented Feb 8, 2012 at 16:41
possible duplicate of How to read large XML file consisting of large number of small items efficiently in Java? — Lukas Eder
– Lukas Eder, Commented Feb 8, 2012 at 16:46
Also similar: stackoverflow.com/questions/7746950/…, stackoverflow.com/questions/2301926/xml-process-large-data, stackoverflow.com/questions/3906892/parse-an-xml-string-in-java, stackoverflow.com/questions/355909/…, etc, etc, etc — Lukas Eder
– Lukas Eder, Commented Feb 8, 2012 at 16:47

helpermethod · Accepted Answer · 2012-02-08 16:44:27Z

5

For large documents (though I wouldn't call your's large) I'd use StAX.

answered Feb 8, 2012 at 16:44

helpermethod

62.8k71 gold badges200 silver badges281 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

FloppyDisk · Accepted Answer · 2012-02-08 16:38:11Z

2

Take a look at the JDOM XML parsing library. It's miles ahead of the native Java parsers, and in my opinion, quite superior.

For the code you provided, you actually have to walk the DOM tree and retrieve elements. See here or the official Java tutorial on working with XML for more information on working with XML documents.

answered Feb 8, 2012 at 16:38

FloppyDisk

1,70316 silver badges26 bronze badges

1 Comment

Spencer Kormos Over a year ago

If the question is just on parsing the 256K file, JDom is good, as well as Dom4J or Xom.

Spencer Kormos · Accepted Answer · 2012-02-08 16:45:44Z

2

You might want to look at a StAX implementation like Woodstox. It lets you pull elements from the parser, instead of the parser pushing data into the app, and lets you pause parsing.

answered Feb 8, 2012 at 16:45

Spencer Kormos

8,4913 gold badges31 silver badges47 bronze badges

Comments

Rob Kielty · Accepted Answer · 2012-05-22 01:50:35Z

2

256Kb is a pretty small file nowadays: yesterday I was working with a 45Gb file which is a factor of 200,000 larger!

It's not clear what your problem is. Any of the normal Java parsing techniques will work perfectly well. Which of them you use depends on why you are parsing the file and what you want to do with the data.

Having said that, many people seem to choose DOM by default because it is so well entrenched. However, more modern object models such as JDOM or XOM are much easier to work with.

edited May 22, 2012 at 1:50

Rob Kielty

8,1928 gold badges43 silver badges52 bronze badges

answered Feb 8, 2012 at 21:41

Michael Kay

165k11 gold badges97 silver badges173 bronze badges

2 Comments

dpsdce Over a year ago

could you please tell what did you use to parse that 45Gb File, actually i need to parse a large XML file of the order of 40 - 50 gb to a TSV or CSV, could you please tell how should i approach this.?

Michael Kay Over a year ago

I was using the streaming facilities in Saxon-EE, documented at saxonica.com/documentation/sourcedocs/streaming.xml

user207421 · Accepted Answer · 2012-05-22 02:22:09Z

0

Don't read the file into a String/StringReader and all that jazz. Parse the file directly via db.parse(new FileInputStream(...)). Reading the file into memory just wastes memory, and time.

answered May 22, 2012 at 2:22

user207421

312k45 gold badges323 silver badges493 bronze badges

Collectives™ on Stack Overflow

How do I parse a large XML file using Java?

5 Answers 5

Comments

1 Comment

Comments

2 Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

1 Comment

Comments

2 Comments

Comments

Linked

Related