I'm doing the following, which basically works. The script tries to insert some lines into a file to rewrite it.
But it is stripping all blank lines and also all line padding. The main problem is that it does not process the last line of the file. I'm not sure why.
while read line; do
<... process some things ...>
echo ${line}>> "${ACTION_PATH_IN}.work"
done < "${ACTION_PATH_IN}"
What can be done to fix this?
while IFS= read -r line; do
## some work
printf '%s\n' "$line" >> output
done < <(printf '%s\n' "$(cat input)")
read not to strip leading and trailing whitespace.read -r prevents backslash at EOL from creating a line continuation."$line") to prevent the shell from doing word splitting and globbing on its value.printf '%s\n' instead of echo because it is reliable when processing values like like -e, -n, etc.< <(printf '%s\n' "$(cat input)") is an ugly way of LF terminating the contents of input. Other constructions are possible, depending on your requirements (pipe instead of redirect from process substitution if it is okay that your whole while runs in a subshell).
IFS has been set for the read, the last printf is not needed, as the read will actually read all the lines verbatim (including spaces), as cat just does nothing with the file contents but copy them to the pipe.
printf ' foo \n\n bar ' | bash -c 'while IFS= read -r line; do printf "<<%s>>\n" "$line"; done' never processes the final non-LF terminated, “bar” line. The printf in the redirect from process substitution is there to add a trailing LF if-and-only-if the original content does not have one; it is not about preserving whitespace but making the last line be processed correctly.read will read a final non-LF-terminated line, but it also returns non-zero, which exits the loop (thus such a line is never processed by the body of the loop).Best yet, use a tool such as awk instead of the shell's while loop. First, awk is meant for parsing/manipulating files so for a huge file to process, awk has the advantage. Secondly, you won't have to care whether you have the last newline or not (for your case).
Hence the equivalent of your while read loop:
awk '{
# process lines
# print line > "newfile.txt"
}' file
One possible reason for not reading the last line is that the file does not end with a newline. On the whole, I'd expect it to work even so, but that could be why.
On MacOS X (10.7.1), I got this output, which is the behaviour you are seeing:
$ /bin/echo -n Hi
Hi$ /bin/echo -n Hi > x
$ while read line; do echo $line; done < x
$
The obvious fix is to ensure that the file ends with a newline.
First thing, use
echo "$line" >> ...
Note the quotes. If you don't put them, the shell itself will remove the padding.
As for the last line, it is strange. It may have to do with whether the last line of the file is terminated by a \n or not (it is a good practice to do so, and almost any editor will do that for you).
