sed on OS X - extract all text that is between square brackets

Question

Given this stream:

[foo] 123 [bar]
[gar] dsa [har] 345
[uf] 88 [gc] 43 [br]

I want to process this with sed (or anything else) so the output would be:

foo bar
gar har
uf gc br

I tried cat myfile | sed -e 's/^.*\[//;s/\].*$//'

But it gives me only the last instance.

My real input is something like:

53f42d4 [the contacts are duplicated] Adding support in picking email verified users [https://trello.com/c/663]
3c454b0 [the contacts are duplicated] splitting contact by phone numbers and emails and changing contact model to contain only 1 email [https://trello.com/c/663]
0e63e5b [we should not let a user confirm his email if we have a user with this confirmed email already] better doc [https://trello.com/c/643]
02671b7 [we should not let a user confirm his email if we have a user with this confirmed email already] preventing updating email if already in used by other user [https://trello.com/c/643]

So I'd like to get for the first line:

the contacts are duplicated https://trello.com/c/663

Answer 1

awk works well for this too: using [ or ] as the field separator, print every even-numbered field:

awk -F '[][]' '{for (i=2; i<=NF; i+=2) {printf "%s ", $i}; print ""}' file

With sed, I'd write

sed -E 's/(^|\])[^[]*($|\[)/ /g' file

Answer 2

This will match anything inside the first (opening) square bracket to the first (closing) square bracket that follows, several times.

$ sed 's/[^[]*\[\([^]]*\)\][^[]*/\1 /g' file
foo bar
gar har
uf gc br

Description:

sed '                      # start a sed script
        s/                 # start a substitute command
        [^[]*              # match all leading characters (except [)
        \[                 # match an explicit [
        \([^]]*\)          # capture text inside brackets.
        \]                 # match the closing ]
        [^[]*              # match trailing text (if any).
        /\1 /              # replace everything matched by the captured text.
        g                  # repeat for all the line.
       ' file              # close script. Apply to file.

This add a trailing space per match. If that must be removed, add a removal at the end:

sed -e 's/[^[]*\[\([^]]*\)\][^[]*/\1 /g' -e 's/ $//' file

If you have GNU grep, this may help (one line per capture).

grep -Po '\[\K[^]]*(?=])'

And, if the above doesn't work, awk could also do it:

awk '{print gensub(/\[([^]]*)\][^[]*/,"\\1 ","g")}' file

Answer 3

An idiomatic way to do that is using look around assertions, see e.g. https://www.regular-expressions.info/lookaround.html, but these are not supported in sed, only in PCRE compliant regular expression processors.

Since Perl should be available on macOS by default, perhaps this is a viable alternative.

Using Perl, you could say

perl -pe 's/.+?(?<=\[)(.+?)(?=\]).+?/$1 /g'

(note that this adds a space at the end of the line)

For an explanation of the pattern, see https://regexr.com/41gi5.

Answer 4

This seems to work:

$ sed -E 's/ [^[][a-zA-Z0-9][^]]/ /g;s/ +/ /g' input | tr -d '[]'
foo bar
gar har
uf gc br

Answer 5

Use:

sed -n '/\[/ { s-[^[]*--; s-\[\([^]]*\)\][^[]*- \1-g; s- --p }'

The algorithm is:

• Ignore lines that do not contain brackets.
• Remove text before first bracket.
• Replace pairs of brackets and optional trailing text by spaces, leaving the text inside the brackets.
• Remove the initial space, leaving only spaces in between.