sed inplace in selective block

Question

I have a dir containing logstash config files these files have filter blocks, and sometimes nested child blocks

need to match whole filter block

filter { any text till final matching closing brace for filter block is found
or 
filter
{ any text till final matching closing brace for filter block is found

till its final ending braces, because there are sometimes many nested blocks that have opening ending braces pair as well.

I'm using grep & xargs + sed to make changes in matched block. need a regex for both grep & sed so changes are only made within the block.

SAMPLE 1

filter {
    json{
        skip_on_invalid_json => true
        source => "message"
    }
    mutate {
        convert => { "rootComponent" => "boolean" }
    }
    date {
        match => ["timestamp","yyyy-MM-dd HH:mm:ss"]
        target => "timestamp"
    }

    ruby {
        code => "
        require 'date'
        week_n = event.get('timestamp').time.strftime '%V'
        month_n = event.get('timestamp').time.strftime '%m'
        year_n = event.get('timestamp').time.strftime '%Y'
        if(week_n == '01' && month_n == '12')
        #year_n = (year_n.to_i + 1) # LINE TO BE UNCOMMENTED
        week_num = year_n.to_s + 'w' + week_n.to_s
        else if (month_n == '01' && week_n.to_i > 50)
        year_n = (year_n.to_i - 1)
        week_num = year_n.to_s + 'w' + week_n.to_s
        else
        week_num = year_n.to_s + 'w' + week_n.to_s
        end
        end
        event.set('[@metadata][week_num]', week_num)
        "
    }

    if([expiresOn] == "N/A") {
        prune {
            blacklist_names => ["log","tags","agent","message","path","@version","host","ecs","input","cloud","expiresOn"]
        }
    } else {
        prune {
            blacklist_names => ["log","tags","agent","message","path","@version","host","ecs","input","cloud"]
        }
    }

}

Line To Edit:

        #year_n = (year_n.to_i + 1) # LINE TO BE UNCOMMENTED

SAMPLE 2

filter{
    json {
        skip_on_invalid_json => true
        source => "message"
    }
    mutate {
      convert => { "rootComponent" => "boolean" }
    }
    mutate {
        add_field => {
          "[@metadata][eventName]" => "%{eventName}"
          "kyc.region" => "N/A"
        }
    }
  ruby {
 code => "
        require 'date'
      week_n = event.get('addKycTimestamp').time.strftime '%V'
      month_n = event.get('addKycTimestamp').time.strftime '%m'
      year_n = event.get('addKycTimestamp').time.strftime '%Y'
      if(week_n == '01' && month_n == '12')
        #year_n = (year_n.to_i + 1) # LINE TO BE UNCOMMENTED

This edit should only occur in filter blocks. Do not edit the line if it appears in other blocks.

I tried below

grep -rl '(?s)filter\s*\{(?:[^{}]+|(?0))*?\year_n =\s*=>(?:[^{}]+|(?0))*?\}' config_dir \
  | xargs -r -I{} sed -i -E '/^[[:space:]]*filter[[:space:]]*\{/,/^[[:space:]]*output[[:space:]]*\{/ s;^[[:space:]]*#([[:space:]]*year_n =\s*=>.*);\1;'

Answer 1

It's more a job for perl whose regexp can more easily find matching pairs. Anyway, the few sed implementations that have a -i option for in-place editing have copied it from perl and in ways incompatible between each other.

If we assume that {/} are always matched even inside "..." strings, that could be

perl -0777 -i -pe '
  s{
    ^ \h* filter \s* ( \{
      (?: [^{}]++ | (?1) ) *
    \} )
  }{
    $& =~ s{ ^ \h* \K \# \h* (year_n \h* =) }{$1}gmrx
  }gmxe' -- your-file

Where:

• -p enables the sed mode where files are processed one record at a time where the equivalent of sed's pattern space is the $_ variable (on which s{pattern}{replacement}flags operates by default, like sed's s).
• -i for in-place editing (since copied by some sed implementations).
• -0777 changes the record separator from the default of newline (like in sed) to some impossible byte value, so the files are processed as a whole (the slurp mode). Same as -g in newer versions of perl.
• then we have a s{pattern}{replacement}gmxe where:
  • x allows adding whitespace (and comments) in the pattern to improve legibility.
  • m makes it so that ^ matches at the start of every line in the subject instead of just at the start of the subject.
  • e is for the replacement to be evaluated as perl code.
  • \s is for any whitespace (well ASCII only by default) including newline, similar to [[:space:]] in POSIX regexp, and \h for horizontal whitespace (ISO8859-1 ones by default, that is space, tab, and non-breaking-space encoded as 0xA0, but importantly not newline; similar to POSIX' [[:blank:]]).
  • ++ is like + but non-backtracking. Can help the matcher not get lost in a backtracking maze if there were unmatched {/}s.
  • The important part in there is (?1) which recalls the regexp in the first (...) capture group so allows for recursive regexps.
  • The replacement applies another s{pattern}{replacement}gmrx to $& which is what was matched by the first regexp with:
    • r returns the result of the substitution instead of applying it in place to $&
    • \K marks the start of what's to Keep from the match, so we don't discard what's matched by what's to the left of it.

To find those files that have such matches so only those be edited, you can do something:

export REGEX='(?xm)
  ^ \h* filter \s* ( \{
    (?: [^{}]++ | (?1) ) *
  \} )'

find config_dir/ -type f -exec perl -0777 -ne '
  print "$ARGV\0" if m{$ENV{REGEX}}' {} + |
  xargs -r0 perl -0777 -i -pe '
    s{$ENV{REGEX}}{
      $& =~ s{ ^ \h* \K \# \h* (year_n \h* =) }{$1}mxgr
    }ge' --

With sed, it's possible to find those matching pairs, but a lot more effort. One approach as seen for instance at my answer to removing braces statements containing nested braces inside is to replace the {/}s starting from the innermost ones in a loop using some escape mechanism until you can match filter{[^{}]*}.

---

Some notes about your attempt:

(?s) (which causes . to also match on newline, but your regexp has no .), (?0) (which recalls the whole regex so doesn't make sense here), \s, *? are all perl regexp operators.

By default grep and sed take Basic Regular Expressions, and Extended Regular Expressions with -E. A few grep implementations (such as GNU grep when built with optional PCRE2 support) can support perl-like regexps with a -P option, but very few sed implementations do.

Some grep and sed implementations support \s and/or *? with -E though. The latter is now specified by POSIX for extended regular expressions since the 2024 edition, but few implementations support it yet as of 2025.

No grep implementation that I know has a slurp mode that matches the regexp against the whole file, though pcre2grep comes close with its -M for multiline mode, and GNU grep with its -z to process NUL-delimiter records (text files are not meant to contain NULs). By default, like sed, they work on one line at a time. GNU sed also has -z for NUL-delimited records, or you can load the whole input into the pattern space programmatically in sed code with a -e :1 -e '$!{N;b1' -e '}' though beware some seds have a relatively low limit on the size of their pattern space.

xargs cannot handle arbitrary file paths unless you use the -0 option (now standard like -r since POSIX2024) on an input with NUL-delimited records.

With -I{} alone, it splits on unquoted newlines, removes trailing blanks, and handles some forms of quotes so would choke on file paths that contain those. It also means running one sed invocation for each file. You're also not using the {} place holder in the command.

It should be grep -rlZ ... | xargs -r0 sed ... (-Z like -r being a non-standard GNU extension).