A shell tool to “tablify” input data containing ANSI escape codes

Question

I have input containing ANSI color codes that I'd like to tablify. I want the output to remain colored, so the tablified output should maintain the ANSI color codes. Hence, naively stripping them away does not meet my requirements.

For example, for this input,

\033[0;32;1mgreen_apple\033[0m 1 100
orange 20 19
pineapple 1000 87
avocado 4 30

The output I expect would be similar to

green_apple 1    100
orange      20   19
pineapple   1000 87
avocado     4    30

In the above output, "green_apple" should be colored according to the color codes from the input i.e. green. I'd like to know how this can be accomplished.

I've tried column, but it doesn't handle ANSI codes. The output of

echo '\033[0;32;1mgreen_apple\033[0m 1 100
orange 20 19
pineapple 1000 87
avocado 4 30' | column -t

is unfortunately

green_apple  1     100
orange                  20    19
pineapple               1000  87
avocado                 4     30

Notice the non-tablification.

Answer 1

I don't think there's such a command, you'd have to do by hand. Something like:

awk '
  {
    nf[NR]=NF
    for (i = 1; i <= NF; i++) {
      f[NR,i] = $i
      gsub(/\033\[[0-9;]*[mK]/, "", $i)
      len[NR,i] = l = length($i)
      if (l > max[i]) max[i] = l
    }
  }
  END {
    for (n = 1; n <= NR; n++) {
      for (i = 1; i < nf[n]; i++)
        printf "%s%*s", f[n,i], max[i]+1-len[n,i], ""
      print f[n,nf[n]]
    }
  }'

Answer 2

For anyone interested, I adapted this awk script from Stéphane Chazelas' answer. You should be able to drop it in your $PATH and run it from the command line.

For example:

> echo -e '\e[32mHello\e[0m,World\nFoo,Bar' | colorcolumn FS=',' OFS=' | '
Hello | World
Foo   | Bar

#!/usr/bin/awk -f
#
# colorcolumn - Like `column`, but works with ANSI colors
#
# Based on this Stack Exchange answer by Stephane Chazelas:
#     https://unix.stackexchange.com/a/121139/259233

{
    nf[NR]=NF
    for (i = 1; i <= NF; i++) {
        cell[NR,i] = $i
        gsub(/\033\[([[:digit:]]+(;[[:digit:]]+)*)?[mK]/, "", $i)
        len[NR,i] = l = length($i)
        if (l > max[i]) max[i] = l
    }
}
END {
    for (row = 1; row <= NR; row++) {
        for (col = 1; col < nf[row]; col++)
            printf "%s%*s%s", cell[row,col], max[col]-len[row,col], "", OFS
        print cell[row,nf[row]]
    }
}

Answer 3

The problem is of course with column not differentiating between printing and non-printing characters, bash gets around this in its prompt (PS1) with its \[ \] escape feature, I don't know of anything else that does exactly that.

I tried a quick and nasty hack, given that your problematic field is at the start of the line, move it to the then end, but you run into similar padding/alignment problems that aren't easily solved using common utilities.

ls -l --color | rev | column -t  | rev   # not a useful solution

If you have the perl HTML::FromANSI module install it has a useful ansi2html script:

ls -l --color | ansi2html -f

That only gets you halfway there, now you have to align HTML output...

There's a straightforward (though slightly heavyweight) solution with HTML as an intermediate: layout of HTML tables is effectively the thing you are trying to do.

This uses Andre Simon's ansifilter, e.g.:

ls -l --color | ansifilter -fH

This converts ANSI sequences to HTML (<span style="..."></span>), then this can be dumped using a text-mode ANSI capable browser like elinks.

If incomplete HTML is a problem, you can optionally run hxclean or htmltidy to clean up the HTML before passing it to a browser.

ls -l --color  | ansifilter -fH | perl table.pl | elinks -dump -dump-color-mode 1 

You should be able to use elinks or w3m for this

The table.pl script splits on whitespace and adds the relevant HTML table tags to achieve the desired formatting:

print "<table>\n";
while (<>) { 
    print "<tr><td>" . 
      join("</td><td>", split(/((?!<[^>]+)\s+(?![^<]+>))/) ) . 
      "</td></tr>\n"; 
}
print "</table>\n";

It doesn't just split on any whitespace, it splits on whitespace that isn't inside '<' '>' angle-brackets so it doesn't break the <span> tag. This is not a good way to parse HTML, but it should suffice for the type of constrained input here.

You may need (I did) to set or override the default colors, either add these to your ~/.elinks/elinks.conf file:

set document.colors.use_document_colors = 1
set document.colors.text = "#000000"
set document.colors.background = "#ffffff"

Make sure to use the latest (0.12.x) elinks, earlier versions do not support ANSI colour output.

Answer 4

Some sed work would do the trick:

• a sed to remove ANSI color codes

  sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g"
  

• a sed to replace space(s) with tabs

  sed -r "s/\s+/\t/g"
  

Finaly:

printf "\033[0;32;1mgreen_apple\033[0m 1 100
orange 20 19
pineapple 1000 87
avocado 4 30" \
| sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" \
| sed -r "s/\s+/\t/g"

Output:

green_apple     1           100
orange          20          19
pineapple       1000        87
avocado         4           30

PS: I honestly tested all this with ls --color output. This is easier than playing with echo.

About removing ANSI color codes:

• From UL: Program to strip color codes
• From SO: Removing color codes with bash
• another source about removing color codes

Answer 5

Assuming that fruit names contain only _ and alphabetic characters.

c='(\e|\\033)\[[0-9;]+m'
sed -r "s/^($c)?([a-zA-Z_]+)($c)?/& \3/" file | column -t | 
sed -r "s/^($c[a-zA-Z_]+$c)( +)([a-zA-Z_]+)/\5\4\1/" |
sed -r "s/^[^ ]+ +//"

Output:

\033[0;32;1mgreen_apple\033[0m  1     100
orange       20    19
pineapple    1000  87
avocado      4     30

---

To (test) display the output in colour, just printf the pipeline via shell command substiturion, eg:

c='(\e|\\033)\[[0-9;]+m'; printf "$(
sed -r "s/^($c)?([a-zA-Z_]+)($c)?/& \3/;" file | column -t | 
sed -r "s/^($c[a-zA-Z_]+$c)( +)([a-zA-Z_]+)/\5\4\1/" |
sed -r "s/^[^ ]+ +//" )"

Output with colour – only green_apple shows in colour.

green_apple  1     100
orange       20    19
pineapple    1000  87
avocado      4     30