awk printing field in the wrong column

Question

When I run the below command, it outputs a list of columns as follows:

# rancher clusters
CURRENT   ID        STATE     NAME                           PROVIDER  
*         abcd      active    test-cluster                   Imported
          efgh      active    prod-cluster                   Imported
          xyzd      active    dev-cluster                    Imported

When I try to print the NAME column, test-cluster is not returned in the results

# rancher clusters | awk '{print $3}'
STATE
active
prod-cluster
dev-cluster

You can see that parts of the STATE column show in the printed NAME column.

When I print the fourth column, test-cluster is returned along with NAME

# rancher clusters | awk '{print $4}'
NAME
test-cluster
Imported
Imported

Why isn't awk returning values from the exact column that is printed? What could cause values from one column to be returned in another column? My expectation is when I print $3, I will get all of the results in the fourth column (NAME).

Answer 1

You are getting $3 - however, by default, awk ignores leading and trailing whitespace when breaking a record up into fields.

You should be able to force it to treat the leading whitespace as an empty field by setting the field separator explicitly ex.

$ cat clusters | awk -F'[ \t]+' '{print $3}'
STATE
active
active
active

whereas

$ cat clusters | awk '{print $3}'
STATE
active
prod-cluster
dev-cluster

Answer 2

awk splits the strings based on a separator (default is one or several whitespaces, i.e. tabs or spaces). To make it clearer, here is your data with the data delimited by |

CURRENT|ID|STATE|NAME|PROVIDER
*|abcd|active|test-cluster|Imported
efgh|active|prod-cluster|Imported
xyzd|active|dev-cluster|Imported

As you can see above, on the first line, $1 is CURRENT, $2 is ID, and so on. On the second line, $1 is *, $2 is abcd, etc. However, on the third line, $1 is efgh, which corresponds to the ID column. Since there is nothing except whitespace in the CURRENT column, it is ignored by awk, and therefore $3 shows the NAME for lines 3 and 4.

What you need to do is to strip off the first column before you send it to awk. The following command should do that for you.

rancher clusters | cut -b 8- | awk '{ print $3 }'

The cut command in the pipeline above will delete the first 7 bytes in every line, and print the rest to STDOUT. That will result in awk not getting confused by different number of columns on every line.

Answer 3

awk by default separates records into fields based on contiguous sequences of white space and ignores any leading trailing white space. Since your first "field" is sometimes empty to awk it doesn't exist. Given that, there's a few ways to print the NAME column, the simplest being to count fields from the end of the record rather than the start of it:

$ awk '{print $NF}' file
PROVIDER
Imported
Imported
Imported
$
$ awk '{print $(NF-1)}' file
NAME
test-cluster
prod-cluster
dev-cluster
$
$ awk '{print $(NF-2)}' file
STATE
active
active
active
$
$ awk '{print $(NF-3)}' file
ID
abcd
efgh
xyzd
$
$ awk '{print (NF>4 ? $(NF-4) : "")}' file
CURRENT
*


$

You need to do some math on that last one so $(NF-4) doesn't cause $0 to be printed when NF is 4. You can calculate the intended number of fields rather than hard-coding 4 by counting how many fields are in the header line:

$ awk 'NR==1{max=NF-1} {print (NF>max ? $(NF-max) : "")}' file
CURRENT
*


$

Answer 4

The problem is that sometimes the output of your rancher command has 5 fields per line, and sometimes it has 4.

The following awk one-liner will print the correct field depending on how many fields (using awk's built-in variable NF) any given input line has:

$ awk 'NF==5 {print $4}; NF==4 {print $3}' clusters.txt
NAME
test-cluster
prod-cluster
dev-cluster

Alternatively, if you can be certain that there will be no blank lines, you could use:

$ awk '{print $(NF-1)}' clusters.txt 
NAME
test-cluster
prod-cluster
dev-cluster

If there is a blank line, you'll get an error message like this:

awk: cmd. line:1: (FILENAME=clusters.txt FNR=5) fatal: attempt to access field -1

That can be avoided with:

awk 'NF>=1 {print $(NF-1)}' clusters.txt