How to use Javascript to read local text file and read line by line?

Using ES6 the javascript becomes a little cleaner

handleFiles(input) {

    const file = input.target.files[0];
    const reader = new FileReader();

    reader.onload = (event) => {
        const file = event.target.result;
        const allLines = file.split(/\r\n|\n/);
        // Reading line by line
        allLines.forEach((line) => {
            console.log(line);
        });
    };

    reader.onerror = (event) => {
        alert(event.target.error.name);
    };

    reader.readAsText(file);
}

Here's a function from the MDN docs that shows you how to use a ReadableStream to read a File line-by-line. This example uses fetch, but if you already have a File, you can call stream() and getReader() instead.

async function* makeTextFileLineIterator(fileURL) {
  const utf8Decoder = new TextDecoder("utf-8");
  let response = await fetch(fileURL);
  let reader = response.body.getReader();
  let { value: chunk, done: readerDone } = await reader.read();
  chunk = chunk ? utf8Decoder.decode(chunk, { stream: true }) : "";

  let re = /\r\n|\n|\r/gm;
  let startIndex = 0;

  for (;;) {
    let result = re.exec(chunk);
    if (!result) {
      if (readerDone) {
        break;
      }
      let remainder = chunk.substr(startIndex);
      ({ value: chunk, done: readerDone } = await reader.read());
      chunk =
        remainder + (chunk ? utf8Decoder.decode(chunk, { stream: true }) : "");
      startIndex = re.lastIndex = 0;
      continue;
    }
    yield chunk.substring(startIndex, result.index);
    startIndex = re.lastIndex;
  }
  if (startIndex < chunk.length) {
    // last line didn't end in a newline char
    yield chunk.substr(startIndex);
  }
}

for await (let line of makeTextFileLineIterator(urlOfFile)) {
  processLine(line);
}

You can reference the following code to read the first lines of a file. But note some caveats and observations:

• Why search for the position of the line break? You might want to directly read 512KB(or any other chunk size) as text. But note that unless you read the entire file all at once, you risk breaking a Unicode character at the 512KB boundary. The last several bytes in the chunk might be an incomplete Unicode. When you are slicing the Blob(File) object, you are slicing a byte array instead of a character array. However, if we locate the position of the line break and read up to that location, we know everything that come before it are whole Unicode characters.

• Does this guarantee not reading a whole file? I do not know, but at least from the consumption point of view, I am not touching anything after the chunks I read. If the underlying mechanism on the browser side wants to mobilize the entire file, this is none of my concern and will be the best I can do.

Example code:

/*
    This function is used to scan the first few lines of a file to determine the position of the nth line break. 
    This is useful for large files where we want to avoid reading the entire file into memory.
    Read is done in chunks of 512KB.
*/
async function scanLinePosition(file: File, lines: number): Promise<number> {
    return await new Promise((resolve, reject) => {
        const reader = new FileReader();
        let rowsRead = 0;
        let chunkSize = 512 * 1024; // 512KB
        let totalRead = 0;

        reader.onload = () => {
            const bytes = new Uint8Array(reader.result as ArrayBuffer);
            for (let i = 0; i < bytes.length; i++) {
                if (bytes[i] === 10) {
                    rowsRead++;
                }
                if (rowsRead >= lines) {
                    break;
                }
            }

            totalRead += bytes.length;

            if (rowsRead >= lines) {
                resolve(totalRead);
                return;
            }

            if (bytes.length === chunkSize && rowsRead < lines) {
                reader.readAsArrayBuffer(file.slice(totalRead, totalRead + chunkSize));
            } else {
                resolve(totalRead);
            }
        };

        reader.onerror = (error) => {
            reject(error);
        };

        reader.readAsArrayBuffer(file.slice(0, chunkSize));
    });
}

async function readFileContent(file: File, lines: number) {
    const readLimit = await scanLinePosition(file, lines);

    return await new Promise((resolve, reject) => {
        const reader = new FileReader();
        reader.onload = () => {
            const rows = (reader.result as string).split("\n");
            if (rows.length >= lines) {
                resolve(rows.slice(0, lines));
            } else {
                reject(new Error("File is too short"));
            }
        };

        reader.onerror = (error) => {
            reject(error);
        };

        reader.readAsText(file.slice(0, readLimit));
    });
}

Files can be processed line by line like this:

async function* linesFromFile (file)
{
  let textStream = file.stream ().pipeThrough (
    new TextDecoderStream ('utf-8', { fatal: true }));

  let regex = /\r\n?|\n/g;
  let remainder = '';

  for await (let text of textStream)
  {
    text = remainder + text;
    let li = 0;

    regex.lastIndex = remainder.length;
    for (let m; m = regex.exec (text);)
    {
      yield text.slice (li, m.index);
      li = regex.lastIndex;
    }

    remainder = text.slice (li);
  }

  yield remainder;
}

Usage:

for await (let line of linesFromFile (file))
  console.log (line);