reading file java.lang.OutOfMemoryError

Question

Trying to find a word in a large file. File is read line by line. When reading the way redLine exception is thrown. Are there any way around this? You can read it on the floor as a string?

for(String line; (line = fileOut.readLine()) != null; ){
                    if(line.contains(commandString)) 
                        System.out.println(count + ": " + line);
                    count++;
                }

java.lang.OutOfMemoryError:

UDP:

this is all my bad code:

static String file = "files.txt";
    static String commandString = "first";
    static int count = 1;

    public static void main(String[] args) throws IOException 
    {

        try(BufferedReader fileOut = new BufferedReader(new InputStreamReader(new FileInputStream(file), "Cp1251")) ){


            for(String line; (line = fileOut.readLine()) != null; ){
                    if(line.contains(commandString)) 
                        System.out.println(count + ": " + line);
                    count++;
                }





            System.out.println("before wr close :"  + Runtime.getRuntime().freeMemory());
            fileOut.close();

        }catch(Exception e) {
            System.out.println(e);
        }
    }

That shouldn't be a for loop. But anyway, what is fileOut, how is it defined and opened? Where does the file come from, and are you sure it's properly broken into lines and not a data/binary file? — RealSkeptic
– RealSkeptic, Commented Feb 8, 2015 at 20:32
You should show us the stack trace, as well as supporting code related to the line where the error occurs. The loop you've provided shows that the line field is overwritten repeatedly, which would not result in OOM. — MarsAtomic
– MarsAtomic, Commented Feb 8, 2015 at 20:33
Why don't you break the for loop when found the commandString? — Chong Tang
– Chong Tang, Commented Feb 8, 2015 at 20:37
Um, no. Don't give lengthy information in comments. Add it as an edit to your original question, properly formatted (though you can draw someone's attention using a comment). Comments are a bad place to put code. — RealSkeptic
– RealSkeptic, Commented Feb 8, 2015 at 20:41
Check that the context has no leaks: especially when repeated. That the fileOut is closed. That fileOut is a BufferedReader of sufficient large size (64*1024). The file seems to contain no line endings or large lines. — Joop Eggen
– Joop Eggen, Commented Feb 8, 2015 at 20:42

Gren · Accepted Answer · 2015-02-09 18:48:16Z

Searching for a word, you can read the file bytewise without holding more than a single byte of the file in memory. Read byte by byte and every time, a byte is equal to the first byte of the searched word, start a second loop and read the following bytes and check if the next byte is equal to the next byte in the word and so on. To give you an example, I have modified an sample to your needs.
I've omitted on the output of the file, because I don't know, if you want to output all lines or only those which contains your keyword and the latter might be as problematic as reading the code line by line.

static String fileName = "files.txt";
static byte[] searchString = { 'f', 'i', 'r', 's', 't' };
static int count = 0;
static long position = 1;
public static void main(String[] args) throws IOException {

    try (FileInputStream file = new FileInputStream(fileName)) {
        byte read[] = new byte[1];
        outerLoop: while (-1 < file.read(read, 0, 1)) {
            position++;
            if (read[0] == searchString[0]) {
                int matches = 1;
                for (int i = 1; i < searchString.length; i++) {
                    if (-1 > file.read(read, 0, 1)) {
                        break outerLoop;
                    }
                    position++;
                    if (read[0] == searchString[i]) {
                        matches++;
                    } else {
                        break;
                    }
                }
                if (matches == searchString.length) {
                    System.out.println((++count)+". found at position "+ (position-matches));
                }
            }

        }
        file.close();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

Collectives™ on Stack Overflow

reading file java.lang.OutOfMemoryError

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related