How to retrieve data from a logfile where timestamp can be followed by multiline log

Question

I have a logfile that I am trying to 'grep' data out of via a bash script. The data I am specifically after is all lines between two timestamps (including the top timestamp) that have the pattern "ERR-" and include an empty line after each entry for readability.

example logfile:

Tue May 24 21:22:12 2022
ERR-0045 Lock detected in /tmp/file.lck
Tue May 24 21:44:12 2022
Errors in file /tmp/filename01.trc:
ERR-0001: Error detected. /tmp/filename.log
Tue May 24 21:47:25 2022
im some output
Tue May 24 21:47:25 2022
im some output too
im some output aswell
Tue May 24 21:48:03 2022
Errors in file /tmp/filename09.trc:
ERR-0100: error
ERR-0050: failure of sorts.
ERR-0052: line 3421
Tue May 24 21:49:07 2022
Completed process xyz

So my desired output would look like :

Tue May 24 21:22:12 2022
ERR-0045 Lock detected in /tmp/file.lck
    
Tue May 24 21:44:12 2022
Errors in file /tmp/filename01.trc:
ERR-0001: Error detected. /tmp/filename.log
        
Tue May 24 21:48:03 2022
Errors in file /tmp/filename09.trc:
ERR-0100: error
ERR-0050: failure of sorts.
ERR-0052: line 3421

I have tried using combinations of sed/awk/cat without much success. Where I am having trouble is:

1. It's not always two lines before ERR- which contains the timestamp
2. There can be multiple ERR-'s in one timestamp block
3. The date will obviously change so i don't want to hard code that in, although the format won't change.

Thanks in advance

Answer 1

Script tst.awk:

function print_r() { if (e) print r; r = ""; e = 0 }

/^([[:alpha:]]{3} ){2}[[:digit:]]{1,2} [[:digit:]]{2}(:[[:digit:]]{2}){2} [[:digit:]]{4}$/ {
    print_r()
}
/^ERR-/{ e = 1 }
{ r = r ORS $0 }
END{ print_r() }

Usage and output:

$ awk -f tst.awk file

Tue May 24 21:22:12 2022
ERR-0045 Lock detected in /tmp/file.lck

Tue May 24 21:44:12 2022
Errors in file /tmp/filename01.trc:
ERR-0001: Error detected. /tmp/filename.log

Tue May 24 21:48:03 2022
Errors in file /tmp/filename09.trc:
ERR-0100: error
ERR-0050: failure of sorts.
ERR-0052: line 3421

You can use your own expressions for the timestamp or the error. Here is a whole line matching of the presented format (no date validation). And "ERR-" at the beginning of a line.

---

Notes:

• We define a function to print a record (r) if error found (e). Also to reset both variables after printing. For awk, variables that have not been initialised with a value are evaluated to the empty string or zero.

• When the regular expression for the date is matched, we call this function. To finish about previous log record and to start holding the new record. Because the record is a multiline log, we don't know yet if we have to print it or not.

• When the error pattern is matching, we set e.

• For every row, we append the row to the existing record, separated by the ORS, the output record separator, default newline. Also, the empty line between output lines is placed at the beginning of r, r is always the empty string when we are here for a new timestamp.

• At the END we call the function again, because the last record is still held.

Answer 2

Using Raku (formerly known as Perl_6)

~$ raku -e 'my @a=slurp; @a.=split(/ <?after ^^ ERR \V* > \n <?before ^^ Tue > /); \
            put $_.subst(/^ <(.+)> ^^ Tue/) ~ "\n" if /^^ ERR / for @a;'   file

The problem seems to be that you have some timestamped records with ERR lines and some without, and you'd like to discard the latter.

Above, the file is slurped into @a array. Then @a is split on \n between records <?after ^^ ERR \V* > ending with a line that starts with ERR followed by zero-or-more \V non-vertical whitespace characters, and <?before ^^ Tue > before a line starting with Tue.

In the second statement output(…) is refined. The @a elements are individually (and automatically) represented by $_ in the preamble to the for clause. Each element is if /^^ ERR/ conditionally-tested to make sure they contain a line starting with ERR: if so, the $_ element is $_.subst(/^ <(.+)> ^^ Tue/) substituted removing all leading characters before an internal start-of-line Tue. Essentially here, .subst without a replacement means to delete the match between <(…)> capture markers. To separate individual records, $_.subst(…) ~ "\n" elements are ~ tilde concatenated with a record-separating "\n" newline.

Sample Input:

Tue May 24 21:22:12 2022
ERR-0045 Lock detected in /tmp/file.lck
Tue May 24 21:44:12 2022
Errors in file /tmp/filename01.trc:
ERR-0001: Error detected. /tmp/filename.log
Tue May 24 21:47:25 2022
im some output
Tue May 24 21:47:25 2022
im some output too
im some output aswell
Tue May 24 21:48:03 2022
Errors in file /tmp/filename09.trc:
ERR-0100: error
ERR-0050: failure of sorts.
ERR-0052: line 3421
Tue May 24 21:49:07 2022
Completed process xyz

Sample Output:

Tue May 24 21:22:12 2022
ERR-0045 Lock detected in /tmp/file.lck

Tue May 24 21:44:12 2022
Errors in file /tmp/filename01.trc:
ERR-0001: Error detected. /tmp/filename.log

Tue May 24 21:48:03 2022
Errors in file /tmp/filename09.trc:
ERR-0100: error
ERR-0050: failure of sorts.
ERR-0052: line 3421

Certainly, the above is a bare bones implementation (it's meant to be--for readability). To work on dates starting on other days of the week, substitute Tues above with the following:

 [ Mon || Tue || Wed || Thu || Fri || Sat || Sun ] 

(It should be obvious from the code above how to add months to your regex, should you run into problems with specificity).

https://raku.org