How to exclude lines that don't include a string from the output of an awk script?

Question

Completely new to scripting in general, and have been working on a script to go through our logs which contain entries for a variety of different things. What I'm interested in, is the entries that log user activity (logins, clicks, etc on our site).

I've pieced together the following awk script (with a ton of help) which almost works perfectly, except it prints output for lines that don't contain a username. I'd like to exclude them, and haven't been able to find out how to do that (I assume it's something simple).

Here is the code:

awk '
{       split($3, d, "@")
        match($0, /"username":"[^"]*"/)
        user = substr($0, RSTART + 12, RLENGTH - 13)
        c[d[1] OFS user]++
}
END {   for(i in c)
                printf("%4d %s\n", c[i], i)
}' mycompany.log | sort -k3,3 -k2,2

I'm just not sure how to exclude all lines that do not contain username from the output.

Here is a sample of a line that contains a username:

qtp111659197-5776 - 05-26@09:37:34:240 INFO  (TimingInfoProxy.java:41)     - com.mycompany.api.ApiHandler-0>getUniqueDataBySource(data,{"has_values":false,"last_event_triggered":"","user_info":{"username":"[email protected]","orgid":"69d467a7-9786-47e1-9c12-bb40f9bfc65d","ip":"127.0.0.1"},"date_range":{"min_date":"","start_date":"","end_date":"","trending_start_date":"","trending_end_date":""},"terms":{"and_filtering":[]}},)

And here's a short example of a line that doesn't:

main - 05-22@10:05:21:387 INFO  (ContextLoader.java:313)     - Root WebApplicationContext: initialization completed in 9519 ms

In what way exactly do they 'not contain' a username? is the "username": keystring missing, or is there an empty quoted value like "username":""? or something else? Please edit your post to include sample entries both with and without the username. — steeldriver
– steeldriver, Commented May 28, 2015 at 16:37
I edited it to include samples, but basically yes the "username": keystring is missing in lines that don't include usernames. — zoo_live_crew
– zoo_live_crew, Commented May 28, 2015 at 16:54
So basically the match part of the code tells it to find the regexp "username":"[^"]*" which in terms of our logs is [email protected] Isn't there something simple I can put in there that basically tells it not to return any output if it can't find regexp "username":"[^"]*" in any given line of input? (sorry I'm very new to this) — zoo_live_crew
– zoo_live_crew, Commented May 28, 2015 at 22:20

DanL4096 · Accepted Answer · 2015-05-28 17:03:35Z

1

Since you're already piping to other shell commands, I'd just use grep.

awk '
{
...
}' mycompany.log | grep -f '"username:"' | sort -k3,3 -k2,2

(Note that I'm using -f because the username tag never varies.)

Edit: that's the most obvious way. An arguably "better" way would be to use awk itself, e.g.

awk '
/\"username\":/ {
...
}' mycompany.log | sort -k3,3 -k2,2

edited May 28, 2015 at 17:03

answered May 28, 2015 at 16:56

DanL4096

8517 silver badges16 bronze badges

Add a comment |

Peter.O · Accepted Answer · 2015-06-29 10:54:14Z

0

Your code isn't checking the result of match. The following fixes it.

RSTART is set by invoking the match function. Its value is the position of the string where the matched substring starts, or zero if no match was found.

if(RSTART){
   user = substr($0, RSTART + 12, RLENGTH - 13)
   c[d[1] OFS user]++
}

edited Jun 29, 2015 at 10:54

answered Jun 29, 2015 at 10:47

Peter.O

33.8k32 gold badges120 silver badges167 bronze badges

Add a comment |

Stack Exchange Network

How to exclude lines that don't include a string from the output of an awk script?

2 Answers 2

You must log in to answer this question.

Hot Network Questions

How to exclude lines that don't include a string from the output of an awk script?

2 Answers 2

You must log in to answer this question.

Related

Hot Network Questions