How to grep with specific format and get occurrences in Linux

Question

I have logfiles and the lines are as follows,

2022-05-21 23:59:59,2406,842,[75000000,074],passed
2022-05-21 23:59:59,2410,841,[750000,076],passed
2022-05-21 23:59:59,3002,892,[700000,78],passed

Are there any method to grep the 75 ,70 as well? I have tried as below, but it is not working. I need the occurrences as well.

cat 20220521log|grep -E "2022-05-21 23|75" -C

Update:

Each and every log contains different timesamps and numbers as above mentioned. I need to check how many occurences can be found in each file according to my pattern. Lets take 20220521 log file I need to check in how many line contains which starts the number field as 75. All other fields are same as before.

2022-05-21 23:59:59,2406,842,[75000000,074],passed //should take as one occurence
2022-05-21 23:59:59,2406,842,[00000000,074],passed //should not consider
2022-05-21 23:59:59,2406,842,[754324000,074],passed //should take as one occurence.

Answer 1

There's no need to invoke several programs for that, perl (and probably also awk/python/...) can do it all:

perl -a -F'' -e 'BEGIN { print "status  count\n" } $a = join "",(@F[30,31]); next unless ($a == 70 or $a == 75); $b{$a}++; END { for (keys %b) { print "$_      $b{$_}\n" } }' < 705361.log

(705361 is the id of the question, just a convention I use when creating files/directories to test commands before putting them here)

Answer 2

If you need to get all the occurrences as well as the count, you could simply do something like:

grep '^2022-05-21.*\[75' logfilename | tee >(wc -l)

This will print all lines starting with 2022-05-21 and has [75 later on the line. (I assume each line only has one number field starting in brackets.) Then on the last line of output it will print the count (by having tee send a duplication of the output to be counted by wc).

If each day has its own file already, you can leave out ^2022-05-21.*; if you only need the count and not the lines, you can remove | tee >(wc -l) and just use grep -c (lowercase c).

Answer 3

Perhaps you want something like:

<your-file grep -Po '^\d\d\d\d-\d\d-\d\d \d\d(?=:\d\d:\d\d,\d+,\d+,\[75)' |
  uniq -c

For a count of lines whose fourth field starts with [75 per hour (assuming the lines are in chronological order).

Answer 4

Using awk

$ awk -v i=1 '/^2022-05-21[^[]*\[(75|70)/{$(NF+1)=i++; print}' input_file
2022-05-21 23:59:59,2406,842,[75000000,074],passed //should take as one occurence 1
2022-05-21 23:59:59,2406,842,[754324000,074],passed //should take as one occurence. 2

Answer 5

code:

cat 20220521log | (echo "status count" ; awk -F "," '{list[substr($4,2,2)]++} END {for(i in list){print i, list[i]}}') | column -nt

result :