Extract information using sed regex

Question

I am using sed regex to extract some information from a log file in order to use it further for analysis. I created a below command but it wont works for me.

sed -e 's/$[0-9] [0-9]*.[0-9]*.[0-9]*$[^@]* ([^@]*@[^[:spa ce:]]*).*F=<$[^ ]*$>.*I=[\([0-9]\+\.[0-9]\+\.)].*$/\1\t\2/' logs

Logs:

2017-02-13 10:31:55 1cd9Ev-003XiE-Sx ** [email protected] F=<[email protected]> R=dkim_lookuphost T=dkim_remote_smtp H=ah2.inboundmx.com [216.82.242.115] I=[147.75.228.64] X=TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256 CV=yes DN="/C=US/ST=California/L=Mountain View/O=Symantec Corporation/OU=Symantec.cloud CN=mail132.messagelabs.com": SMTP error from remote mail server after end of data: 553-Message filtered. Refer to the Troubleshooting page at\n553-http://www.symanteccloud.com/troubleshooting for more\n553 information. (#5.7.1)

2017-02-14 10:01:40 1cd9Ev-003XiE-Sx ** [email protected] F=<[email protected]> R=dkim_lookuphost T=dkim_remote_smtp H=ah2.inboundmx.com [216.82.242.115] I=[14.176.22.221] X=TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256 CV=yes DN="/C=US/ST=California/L=Mountain View/O=Symantec Corporation/OU=Symantec.cloud CN=mail132.messagelabs.com": 501 Connection rejected by policy. Refer to the Troubleshooting page at\n501-http://www.symanteccloud.com/troubleshooting for more\n501 information. (#5.7.1)

I wanted to extract the following fields from above logs:

Timestamp            EmailTo:           EmailFrom:      IPAddress:      ErrorCodes:
2017-02-13 10:31:55 [email protected]  [email protected]  147.75.228.64   553
2017-02-14 10:01:40 [email protected] [email protected]  14.176.22.221   501

yeah, run awk '{t=$0;sub(/.*\\n/, "", t);sub(/ .*/, "", t);print $1, $2, $5, substr($6, 4, length($6)-4), substr($11, 4, length($11)-4), t}' ... the t, sub, sub extracts the error code, the rest is self-explanatory - print the respective fields except for $6 and $11 where it extracts only part of the field and prints the result — don_crissti
– don_crissti, Commented Feb 14, 2017 at 19:13
I am not having any other answer except thanking you. If you don't mind let me know how can i master in awk and sed ? — blaCkninJa
– blaCkninJa, Commented Feb 14, 2017 at 19:21
well, read the manuals (both gnu sed and gnu awk manuals can be downloaded as pdf) & practice at the same time; just like with anything else in life, you have to crawl before you walk :) ... so start with simple tasks and work your way up as your skills improve — don_crissti
– don_crissti, Commented Feb 14, 2017 at 19:27

Costas · Accepted Answer · 2017-02-15 08:35:28Z

2

Other idea instead of extracting fields needed is to remove extra:

sed '
    s/[^: ]*\s\*\*\s//
    s/F=<//
    s/>.*I=\[/ /
    s/\].*more\\n/ /
    s/\sinf.*//
    ' log.file

first command remove 1cd9Ev-003XiE-Sx **
second — F=<
third — > R=dkim_lookuphost T=dkim_remote_smtp H=ah2.inboundmx.com [216.82.242.115] I=[

and so on…

edited Feb 15, 2017 at 8:35

answered Feb 14, 2017 at 19:27

Costas

15k24 silver badges38 bronze badges

@rlinux57 I do not understand which further explanation needed but here you are.

Costas
– Costas

2017-02-15 08:44:40 +00:00
Commented Feb 15, 2017 at 8:44

Add a comment |

Stack Exchange Network

Extract information using sed regex

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Extract information using sed regex

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions