16

I want to get just e-mail addresses that end in "@xyz.nl" from my mail logfile. To achieve this I do:

# tail -f /var/log/mail.log | grep --i --line-buffered "@xyz.nl" | cut -d '@' -f 1 | cut -d '<' -f 2

The --line-buffered with grep is necessary because it will otherwise buffer its output because the pipe is not considered a terminal. Grep will output lines like these:

Aug 29 11:56:01 localhost postfix/smtp[4124]: 05491500123: to=<[email protected]>, relay=123.456.123.456[123.456.123.456]:25, delay=2, delays=0.4/0/0.4/1.2, dsn=2.0.0, status=sent (250 2.0.0 u7T9twxN074009 Message accepted for delivery)

The first cut then makes:

Aug 29 11:56:01 localhost postfix/smtp[4124]: 05491500123: to=<someone

The second cut should generate:

someone

However it seems that cut is also buffering. If I start the command with cat instead of tail -f I get all relevant results (in the prefered format) from the log file. But I need the results from the log file in real time.

I tried using unbuffer for this:

# tail -f /var/log/mail.log | grep --i --line-buffered "@xyz.nl" | unbuffer cut -d '@' -f 1 | cut -d '<' -f 2

Also tried:

# unbuffer tail -f /var/log/mail.log | grep --i --line-buffered "@xyz.nl" | unbuffer cut -d '@' -f 1 | cut -d '<' -f 2

...which should remove the 4K buffering from the first cut. However, this doesn't work. I know it is buffering because if I grep for our local domain it gets a lot more hits, the buffer is filled sooner and output is generated earlier (in 4K batches).

So my question is: how do I unbuffer cut?

Related: I know sed and (g)awk can deliver the e-mail addresses to me. I have been trying but as yet without any result. Answers using sed or (g)awk are welcome and may solve my direct issue but I remain interested in the nominal answer of the question how to unbuffer the cut command. The cut command doesn't speak of (un)buffering.

4
  • awk -F'[><@]' '/@xyz.nl/ {print $2}'... Commented Sep 2, 2016 at 7:02
  • 1
    try grep -oP '[^<]+([email protected])' (along with your other grep options as needed) Commented Sep 2, 2016 at 7:08
  • Both the awk and grep options above work like a charm! Commented Sep 2, 2016 at 7:31
  • Still unanswered: Why does’t unbuffer work here? Commented Dec 12, 2020 at 16:42

2 Answers 2

32

If you're on a system using GNU Coreutils (almost any Linux), you can try stdbuf:

… | stdbuf -oL cut -d '@' -f 1 | …

-oL makes it line buffered, which seems like what you want.

2
  • Does not change anything for me for some reason. :( Commented Jun 29, 2017 at 10:32
  • 1
    @panzi Feel free to ask your own question, linking to this one. Please include full details (exact command you're running if possible, OS, distro, and version, etc.) Hopefully someone will be able to figure out why it doesn't work for you. Commented Jun 29, 2017 at 16:23
0

I tried stdbuf -oL to no avail. stdbuf -o0 seemed to work though.

2
  • Please add some context. What was the exact command you tried? What operating system are you using? Commented Jul 1 at 14:44
  • Sorry, yes, I was tailing a Kafka topic with very long records on an Alma9.2 linux system, and just wanted to see the beginning of each record. Just using cut, I never saw anything of the latest record. So I piped the output through the command "stdbuf -o0 cut -c -240", and the latest record showed as desired. Commented Jul 2 at 0:00

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.