Quickest way to delete redundant lines in bash, or delete last line if string is #

Question

Looking for an elegant way, when at the end of a bash script to cleanup the resulting file if:

1. the first character of the line below the line to be deleted is # or
2. # is the first character of the last line in the file.

File could look like this (line numbers of reference are in [] but are not in the file itself):

[1] # foo
[2] bar1
[3] bar2
[4] bar3
[5] # foob
[6] #ar
[7] ar1
[8] ar2
[9] ar3
[10] #

In the above example I'd like to delete lines 5 and 10. I do not want any line starting with # to be followed by a line starting with # and I do not want the last line of the file to start with #.

EDIT: Expected output:

[1] # foo
[2] bar1
[3] bar2
[4] bar3
[5] #ar
[6] ar1
[7] ar2
[8] ar3

Teaching myself bash as I go and am super confused by sed and awk syntax, but I suspect they are the answer here.

Answer 1

I think what you want is

sed -e '${/^#/d}' -e '$!N;/^#.*\n#/D' -e 'P;D' file

• ${/^#/d} address the last line $; if it matches ^# then delete it

• $!N ... P;D maintain a rolling two-line buffer

• /^#.*\n#/D if there is a # both at the start of the pattern and after the newline, delete up to the newline and begin a new cycle

Answer 2

Your requirements can be coded in sed as shown.

sed -e '
  /^#/!b
  $d;N
  /\n#/D
' file

• Any non interesting line is taken to stdout, meaning printed: /^#/!b

• interesting line is one that begins with a #

• last line which happens to be interesting is deleted: $d

• Please note that a non interesting line CANNOT come to 2nd sed code line.

• in case the interesting line is not the last line of the file, we append the next line to it: N

• if the next line is interesting as well, remove the previous interesting line AND go to the top of the sed script using the existing pattern space: /\n#/D

You should refer to the sed manual for details on the various sed commands used.

Answer 3

You can use sed multi-line techniques

sed -E 'N;s/(^#.*)(#.*)/\2/;${s/(.*)(#.*)/\1/}' file

N command appends a newline and the next line to the pattern space.

for lines with pattern ^#.*#.* removes the first line.

${} for last line checks if the pattern .*#.* matches, removes the last part.

Answer 4

sed -n '/^#/{:a;h;n;//b a;H;g;};p' file

Option -n disables automatic printing, lines will be printed via p action. /^#/{ ... } – on lines beginning with #, perform a group of actions. :a ... b a sets up a loop, in that loop:

• h replaces the hold space with the pattern space (i.e. saves the current line to a buffer).
• n reads the next line of input into the pattern space (or quits there is no more input).
• //b a – branch to :a if pattern space matches /^#/. In sed, an empty regular expression is shorthand for apply the last used regex.

Once loop is finished: H;g – append current pattern space to the hold space and replace pattern space with hold space. p at the end prints the result from the loop, and every other non-# line.

Result: only the last of any group of # lines is printed, and no # lines before the end.

# foo
bar1
bar2
bar3
#ar
ar1
ar2
ar3