I can't use grep on all of the input since it's a stream. I can accumulate a chunk of stream and use grep on it...

Are you aware that pipelines block? If you pipe something to grep and all input is not available, grep will wait until it is available and then continue as if the input was there all along.

$ ( echo a1; echo b1; sleep 5; echo a2 ) | grep 'a.'
a1
a2

EDIT: How pipelines work, for example with cmd1 | cmd2 is that both programs will start at the same time, with an e.g. 65,536-byte "chunk buffer" between them. When cmd2 tries to read and that buffer is empty, it will wait for a chunk to be available. When cmd1 tries to write and that buffer is full, it will wait until cmd2 reads it.

From what I can read, there is no need to cut the input into chunks and pass them to grep separately. That's already done automatically.

I can't use grep on all of the input since it's a stream. I can accumulate a chunk of stream and use grep on it...

Are you aware that pipelines block? If you pipe something to grep and all input is not available, grep will wait until it is available and then continue as if the input was there all along.

$ ( echo a1; echo b1; sleep 5; echo a2 ) | grep 'a.'
a1
a2

EDIT: How pipelines work, for example with cmd1 | cmd2 is that both programs will start at the same time, with an e.g. 65,536-byte "chunk buffer" between them. When cmd2 tries to read and that buffer is empty, it will wait for a chunk to be available. When cmd1 tries to write and that buffer is full, it will wait until cmd2 reads it.

From what I can read, there is no need to cut the input into chunks and pass them to grep separately. That's already done automatically.

EDIT2: grep should also print the results as soon as it finds them in the stream. There is no need for the stream to finish before you can get your results.

I can't use grep on all of the input since it's a stream. I can accumulate a chunk of stream and use grep on it...

Are you aware that pipelines block? If you pipe something to grep and all input is not available, grep will wait until it is available and then continue as if the input was there all along.

$ ( echo a1; echo b1; sleep 5; echo a2 ) | grep 'a.'
a1
a2

From what I can read, there is no need to cut the input into chunks and pass them to grep separately.

I can't use grep on all of the input since it's a stream. I can accumulate a chunk of stream and use grep on it...

Are you aware that pipelines block? If you pipe something to grep and all input is not available, grep will wait until it is available and then continue as if the input was there all along.

$ ( echo a1; echo b1; sleep 5; echo a2 ) | grep 'a.'
a1
a2

EDIT: How pipelines work, for example with cmd1 | cmd2 is that both programs will start at the same time, with an e.g. 65,536-byte "chunk buffer" between them. When cmd2 tries to read and that buffer is empty, it will wait for a chunk to be available. When cmd1 tries to write and that buffer is full, it will wait until cmd2 reads it.

From what I can read, there is no need to cut the input into chunks and pass them to grep separately. That's already done automatically.