14

I'm trying to grep live text stream from netcat, but it doesn't work for me:

netcat localhost 9090 | grep sender

returns nothing, but I'm sure that it should.

If I redirect the netcat output to a file and add some delays (simulate real environment) - then it works: 

$ (sleep 5; cat netcat_output; sleep 5) | grep sender

{"jsonrpc":"2.0","method":"GUI.OnScreensaverDeactivated","params":{"data": "shuttingdown":false},"sender":"xbmc"}}

I also tried to add --line-buffered but w/o success.

What I do wrong?

Edit:

I noticed the same issue with sed, the output is empty.

But, for instance, hexdump converts text to hex live:

$ netcat localhost 9090 | hexdump -C
00000000  7b 22 6a 73 6f 6e 72 70  63 22 3a 22 32 2e 30 22  |{"jsonrpc":"2.0"|
00000010  2c 22 6d 65 74 68 6f 64  22 3a 22 50 6c 61 79 65  |,"method":"Playe|
00000020  72 2e 4f 6e 50 6c 61 79  22 2c 22 70 61 72 61 6d  |r.OnPlay","param|
00000030  73 22 3a 7b 22 64 61 74  61 22 3a 7b 22 69 74 65  |s":{"data":{"ite|
00000040  6d 22 3a 7b 22 69 64 22  3a 36 2c 22 74 79 70 65  |m":{"id":6,"type|
00000050  22 3a 22 6d 6f 76 69 65  22 7d 2c 22 70 6c 61 79  |":"movie"},"play|
00000060  65 72 22 3a 7b 22 70 6c  61 79 65 72 69 64 22 3a  |er":{"playerid":|
00000070  31 2c 22 73 70 65 65 64  22 3a 31 7d 7d 2c 22 73  |1,"speed":1}},"s|
Improve this question
5
  • 2
    Maybe it gets buffered in the pipe? You could try using stdbuf -o0 netcat localhost 9090 | grep sender (taken from here) Commented Dec 29, 2014 at 23:12
  • Does it work like this ? netcat -z localhost 9090 | grep sender Commented Dec 29, 2014 at 23:21
  • @user43791 the output is still empty, but I'm sure that netcat returned matching string Commented Dec 30, 2014 at 10:39
  • @sputnick this one immediately returns to shell and doesn't wait for an event Commented Dec 30, 2014 at 10:40
  • 1
    There are no new lines in the hexdump, so grep is probably waiting endlessly for a newline. cat works since grep will get an EOF if not a newline, at least. Maybe try awk with { as RS? Commented Dec 30, 2014 at 11:10

6 Answers 6

Reset to default
12

Answer is here: grep not matching in nc output

netcat outputs verbose logs to standard error, so we need to capture errors before we pipe to grep.

$ netcat -zv localhost 1-9000 2>&1 | grep succeeded
Improve this answer
1
  • This answer just made me solution. That was very strange when I run that command without 2>&1, it look like grep didn't work at all. Commented Jan 24, 2018 at 22:31
4

You could use the read command (bash builtin) to force characters to be read one by one :

netcat localhost 9090 | (
    cnt=0
    line=
    while read -N 1 c; do
        line="$line$c"
        if [ "$c" = "{" ]; then
            cnt=$((cnt+1))
        elif [ "$c" = "}" ]; then
            cnt=$((cnt-1))
            if [ $cnt -eq 0 ]; then
                printf "%s\n" "$line"
                line=
            fi
        fi
    done
) | grep sender

This script should print every full output with balancing {and }, but you can change the script to do whatever you want. This script would NOT do well on a benchmark compared to pretty much anything, but it's pretty simple and seems to work for me...

Note that your test sample didn't have matching {and }, so if this is the case of the real input, you might want to change the criteria to print the line.

Improve this answer
1
  • Looks like the mismatching braces is just because he only posted the beginning of the output. Commented Dec 31, 2014 at 19:09
2

I think the issue is the absence of newlines in the netcat output. I can see two workarounds:

  1. Insert a newline every x seconds (with unfortunate consequences if the newline is inserted in the middle of source):

    ( while sleep 1; do echo; done & netcat ... ) | grep source

  2. Use awk with an RS other than newline:

    netcat ... | awk -v RS='}' -v ORS='}' '/source/'
Improve this answer
2
  • Still empty output. To make sure that command is correct I run on dumped stream - cat netcat_output | awk -v RS='}' -v ORS='}' '/sender/' The output was ok: ,"sender":"xbmc"} Commented Dec 30, 2014 at 12:55
  • I just checked first workaround and it works! But as you mentioned it can split the output so it cannot be the final solution. Anyway many thanks! Commented Dec 30, 2014 at 13:56
1

use watch command:

watch 'netcat localhost 9090 | grep sender'
Improve this answer
1
  • It doesn't work too. The same with sed... But, for instance, this: netcat localhost 9090 | hexdump works and replace text to hex live, why? Commented Dec 30, 2014 at 10:32
0

As per @user43791 comment, try disabling buffering in pipe, e.g.:

stdbuf -o0 netcat localhost 9090 | grep sender
Improve this answer
0

In Bash you can use pseudo-device files to open a TCP connection and grep it as usual. For example:

$ grep "sender" < /dev/tcp/example.com/1234

To test it, simply run server which can send some file, like:

$ nc -vl 1234 < /etc/hosts

Then in another terminal run the test to grep the output:

$ grep "127" < /dev/tcp/localhost/1234
127.0.0.1   localhost
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.