2

I want to write a shell command that replaces all newlines from all paragraphs in stdout that match a specific regex with spaces. Here, I'm defining a paragraph to be any text bookended by two or more new lines.

Specifically, I want to locate all paragraphs of text that do not begin with ( or $ and strip these paragraphs of all newlines.

For example, running my script on

Aliquam erat volutpat.  Nunc ( eleifend leo vitae magna.  In (i)yd erat non orci
commodo lobortis.  Proin $ neque massa, cursus ut, gravida ut, lobortis eget,
lacus.  Sed diam.

Hello world.

(Nullam tristique diam
non turpis.



Hello
$again!

$foo
bar

should result in

Aliquam erat volutpat.  Nunc ( eleifend leo vitae magna.  In (i)yd erat non orci commodo lobortis.  Proin $ neque massa, cursus ut, gravida ut, lobortis eget, lacus.  Sed diam.

Hello world.

(Nullam tristique diam
non turpis.



Hello $again!

$foo
bar

Is this possible?

I don't mind if there's collateral damage like adding extra newlines (but I'm also curious if it can be done without collateral damage!).

Improve this question
1
  • Difficult if you're restricted to a single command. Languages like perl or emacs can handle this easily. Commented Jan 5, 2021 at 21:01

3 Answers 3

Reset to default
6

Using any awk in any shell on all Unix boxes:

$ awk -v RS= -v ORS='\n\n' -F'\n' '!/^[($]/{$1=$1} 1' file
Aliquam erat volutpat.  Nunc ( eleifend leo vitae magna.  In (i)yd erat non orci commodo lobortis.  Proin $ neque massa, cursus ut, gravida ut, lobortis eget, lacus.  Sed diam.

Hello world.

(Nullam tristique diam
non turpis.

Hello $again!

$foo
bar

The above will print a blank line after every paragraph, including the last one in the file even if it didn't have one originally. If that's an issue let us know as it's easy to not do so.

Improve this answer
6
  • 1
    @Quasímodo null RS doesn't cause the field separator to be a newline, it causes it to include a newline. It still also includes blank and tab (or whatever else the FS is set to if it's a single char). Commented Jan 6, 2021 at 15:09
  • 1
    Oh, that's it. Quoting verbatim: "The newline shall always be a field separator." So the newline being a field separator does not mean it is the only one. What a tricky wording in the specification! Commented Jan 6, 2021 at 15:13
  • 1
    Right and the current version of the POSIX spec is unfortunately wrong where it says a <newline> shall always be a field separator, no matter what the value of FS is but that's not quite true, it should end with ...if FS is a single char as that's how all awks actually berhave. I'm pretty sure I have a bug report open against the spec about that, let me check... Commented Jan 6, 2021 at 15:14
  • 1
    @Quasímodo Ah, now I remember - I had raised the issue with the gawk providers (see lists.gnu.org/archive/html/bug-gawk/2019-04/msg00029.html) and THEY were going to follow up with the standards folks to get it fixed there. Unfortunately at that point I lost interest and didn't pursue the standard change, I expect it is in the queue somewhere. Commented Jan 6, 2021 at 15:22
  • 1
    @Quasímodo I see that functionality and bug in the spec is now specifically addressed in the gawk manual (gnu.org/software/gawk/manual/gawk.html#Multiple-Line): When RS is set to the empty string and FS is set to a single character, the newline character always acts as a field separator. This is in addition to whatever field separations result from FS....Note that language in the POSIX specification implies that this special feature should apply when FS is a regexp. However, Unix awk has never behaved that way, nor has gawk. This is essentially a bug in POSIX. Commented Jan 6, 2021 at 15:53
5

As extra blank lines don't matter

gawk 'BEGIN {RS=""} !/^[$(]/ {gsub("\n"," ")} {print;print "\n"}'

Explanation. RS="" sets gawk into paragraph mode. !/^[$(]/ matches paragraphs that don't start with ( or $. gsub("\n"," ") changes newlines into spaces. print;print "\n" outputs the data and a newline.

Improve this answer
1
  • 1
    With gawk, you could use {printf "%s%s",$0,RT} to preserve the exact paragraph separation I think? Commented Jan 5, 2021 at 21:57
2

GNU sed

Store the paragraph in the hold space. This method will preserve multiple empty lines.

sed -e '
  /./,/^$/!b
  H;/./{$!d;}
  z;x;s/.//;$!s/.$//
  /^[$(]/!y/\n/ /
  $!G
' file

perl -pl -00e '
  y/\n/ / unless /^[$(]/;
  s/$/\n/ unless eof;
' file

Perl in paragraph mode -00. The consecutive empty lines will be collapsed into one.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.