1

I need to pipe the output of one command to two other commands.

cat filename.txt | tail -n 1
cat filename.txt | wc -l

Since the file is huge I want to avoid reading it twice. I checked tee command but it redirects output to file which I don't want. There are many related posts but did not find anything relevant

0

3 Answers 3

2

If I understood correctly, you want to get the last line of the file along with the line count:

$ cat a
aaa
bb
ccc
$
$ awk 'END{print $0; print NR}' a
ccc
3

Since you need them to save in variables:

$ out=$(awk 'END{print $0"|"NR}' a)
$ last_line=$(echo $out | awk -F"|" '{print $1}' )
$ tot_cnt=$(echo $out | awk -F"|" '{print $2}' )
$ echo $last_line
ccc
$ echo $tot_cnt
3
4
  • yes thats exactly what I want..and i want both in variable Commented Jul 23, 2015 at 10:07
  • how can I save it in variable? Commented Jul 23, 2015 at 10:11
  • I'm not that good in shell..I understand NR=number of records but how awk is getting last record can you please explain? Commented Jul 23, 2015 at 10:36
  • END label instructions are executed after the last line of the file is processed. Hence, inside END when you try to print $0, it ends up printing the last line. Commented Jul 23, 2015 at 10:43
2
{   
    {   tee /dev/fd/3 | wc -l >&4
    }   3>&1 | tail -n1
}   <filename.txt 4>&1

It doesn't avoid reading it twice - but on multicore systems it will likely be concurrent. The output order is in no way guaranteed, however.

Probably the fastest way to simulate what you're asking for, though, is:

dd bs=64k <filename.txt | tail -n1

dd will report its read/writes to stderr and still copy all of the file to stdout.

1

Let the cat sleep, you don't need it.

Use :

tail -n 1 filename.txt
wc -l filename.txt
3
  • 2
    Yes, but this still reads the input file twice which is what the OP is trying to avoid. Commented Jul 23, 2015 at 10:11
  • I can't avoid cat.. Commented Jul 23, 2015 at 10:12
  • 1
    @heemayl: you are perfectly right, and it is not just syntactic sugar. Measuring the performance shows that getting rid of cat is much faster: tail knows how big the file is and can be optimized to check directly at the file end before returning the last line, but it does not know how big stdin is going to be, so it has to wait until cat is done before returning its answer. A similar thing applies to wc, which performs better against a file than against stdin. Commented Jul 23, 2015 at 10:13

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.