Revisions to Perl or sed script to remove a specific content of a file from another bigger file

added 72 characters in body

Source Link

edited Aug 6 at 13:11

35.8k
6
25
60

Assuming you're good with full-record (i.e. matching on the whole multi-line block), full-line regexp matching (see my comment), you could do this with GNU awk for multi-char RS:

$ awk -v RS='^$' 'NR==FNR{RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

or for full-record, full-line literal string matching we could just escape all of the characters from pat to disable any potential regexp metachars:

$ awk -v RS='^$' 'NR==FNR{gsub(/[^^\\]/,"[&]"); gsub(/[\\^]/,"\\\\&"); RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

The above was run using this input:

$ head -50 pat file1
==> pat <==
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13

==> file1 <==
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

Assuming you're good with full-line regexp matching (see my comment), you could do this with GNU awk for multi-char RS:

$ awk -v RS='^$' 'NR==FNR{RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

or for full-line literal string matching we could just escape all of the characters from pat to disable any potential regexp metachars:

$ awk -v RS='^$' 'NR==FNR{gsub(/[^^\\]/,"[&]"); gsub(/[\\^]/,"\\\\&"); RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

The above was run using this input:

$ head -50 pat file1
==> pat <==
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13

==> file1 <==
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

Assuming you're good with full-record (i.e. matching on the whole multi-line block), full-line regexp matching (see my comment), you could do this with GNU awk for multi-char RS:

$ awk -v RS='^$' 'NR==FNR{RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

or for full-record, full-line literal string matching we could just escape all of the characters from pat to disable any potential regexp metachars:

$ awk -v RS='^$' 'NR==FNR{gsub(/[^^\\]/,"[&]"); gsub(/[\\^]/,"\\\\&"); RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

The above was run using this input:

$ head -50 pat file1
==> pat <==
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13

==> file1 <==
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

added 96 characters in body

Source Link

edited Aug 6 at 12:38

Ed Morton

35.8k
6
25
60

Assuming you're good with full-line regexp matching (see my comment), you could do this with GNU awk for multi-char RS:

$ awk -v RS='^$' 'NR==FNR{RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

or for full-line literal string matching we could just escape all of the characters from pat to disable any potential regexp metachars:

$ awk -v RS='^$' 'NR==FNR{gsub(/[^^\\]/,"[&]"); gsub(/[\\^]/,"\\\\&"); RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

The above was run using this input:

$ head -50 pat file1
==> pat <==
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13

==> file1 <==
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

Assuming you're good with full-line regexp matching (see my comment), you could do this with GNU awk for multi-char RS:

$ awk -v RS='^$' 'NR==FNR{RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

or for full-line literal string matching:

$ awk -v RS='^$' 'NR==FNR{gsub(/[^^\\]/,"[&]"); gsub(/[\\^]/,"\\\\&"); RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

The above was run using this input:

$ head -50 pat file1
==> pat <==
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13

==> file1 <==
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

Assuming you're good with full-line regexp matching (see my comment), you could do this with GNU awk for multi-char RS:

$ awk -v RS='^$' 'NR==FNR{RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

or for full-line literal string matching we could just escape all of the characters from pat to disable any potential regexp metachars:

$ awk -v RS='^$' 'NR==FNR{gsub(/[^^\\]/,"[&]"); gsub(/[\\^]/,"\\\\&"); RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

The above was run using this input:

$ head -50 pat file1
==> pat <==
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13

==> file1 <==
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

added 156 characters in body

Source Link

edited Aug 6 at 12:32

Ed Morton

35.8k
6
25
60

Assuming you're good with full-line regexp matching (see my comment), you could do this with GNU awk for multi-char RS:

$ awk -v RS='^$' 'NR==FNR{RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

or for full-line literal string matching:

$ awk -v RS='^$' 'NR==FNR{gsub(/[^^\\]/,"[&]"); gsub(/[\\^]/,"\\\\&"); RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

The above was run using this input:

$ head -50 pat file1
==> pat <==
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13

==> file1 <==
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

Assuming you're good with full-line regexp matching (see my comment), you could do this with GNU awk for multi-char RS:

$ awk -v RS='^$' 'NR==FNR{RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

The above was run using this input:

$ head -50 pat file1
==> pat <==
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13

==> file1 <==
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

Assuming you're good with full-line regexp matching (see my comment), you could do this with GNU awk for multi-char RS:

$ awk -v RS='^$' 'NR==FNR{RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

or for full-line literal string matching:

$ awk -v RS='^$' 'NR==FNR{gsub(/[^^\\]/,"[&]"); gsub(/[\\^]/,"\\\\&"); RS="\n("$0")?"; next} 1' pat file1
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

The above was run using this input:

$ head -50 pat file1
==> pat <==
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13

==> file1 <==
LINE 1 ABCD
LINE 2 EFGA
LINE 3 HCJK
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 4 ABCDH
LINE 5 EFGAG
LINE 6 HCJKD
REMOVE LINE11
  REMOVE LINE12
    REMOVE LINE13
LINE 7 ABCDH
LINE 8 EFGAG
LINE 9 HCJKD

Source Link

answered Aug 6 at 12:20

Ed Morton

35.8k
6
25
60

Loading

Stack Exchange Network

Return to Answer