color terminal side-by-side diffs broken in python

Question

I'm trying to build a python app to quickly generate color side-by-side diffs in python. My problem is that I can generate side-by-side from the linux CLI, but diffs fail under python no matter what command-mangling I've tried (see attempts below).

This is correct behavior if I run the diffs from linux CLI (under wsltty). Note that I get the same output from PuTTY:

This is the typical unwanted behavior when I run the diffs from a python script...

QUESTIONS:

Why are the python attempts below failing to generate side-by-side diffs?
What can I do (under python) to render output as I can from the CLI (screenshot above)?

The following python script contains the attempts I've tried to make python-rendered side-by-side diffs work... in short, I've tried a number of attempts with Popen(), as well as subprocess.run() and os.system()...

# filename: test_ydiff.py
from subprocess import run, Popen, PIPE
import shlex, os, pydoc

TEST_01 = True
TEST_02 = True
TEST_03 = True
if TEST_01:
    cmd_01_input   = "diff -u f_01.txt f_02.txt"
    cmd_01_output  = "ydiff -s"
    proc_01_input  = Popen(shlex.split(cmd_01_input),
                                    stdout=PIPE)
    proc_01_output = Popen(shlex.split(cmd_01_output),
        stdin=proc_01_input.stdout, stdout=PIPE)
    stdout_str, stdin_str = proc_01_output.communicate()
    print(stdout_str.decode('utf-8'))

if TEST_02:
    cmd_02_shell    = "diff -u f_01.txt f_02.txt | ydiff -s"
    proc_02_shell   = Popen(cmd_02_shell, shell=True, stdout=PIPE)
    stdout_str, stdin_str = proc_02_shell.communicate()
    print(stdout_str.decode('utf-8'))

if TEST_03:
    run("/usr/bin/diff -u ./f_01.txt ./f_02.txt|/home/mpennington/venv/py37_u18/bin/ydiff -s")

First text file to be diffed:

# filename: f_01.txt
!
interface Ethernet0/0
 ip address 10.0.0.1 255.255.255.0
 no ip proxy-arp
 no ip unreachables
 ip access-group FILTER_in in
!

Second text file to be diffed:

# filename: f_02.txt
!
interface Ethernet0/0
 ip address 10.0.0.1 255.255.255.0
 ip proxy-arp
 no ip unreachables
!

I'm running Python 3.7.6 under Ubuntu 18... I've got ydiff==1.1 (github: ydiff)

Peter Moore · Accepted Answer · 2020-06-16 21:29:43Z

This is one of those tty setting problems. Sure enough a quick look at ydiff.py shows it inspecting sys.stdout.isatty() (aka os.isatty(2)) and this is used to find out if your stdout fd is piped to some command. You can then decide to strip color format and that is what ydiff is doing to you.

"diff -u f_01.txt f_02.txt | ydiff -s " This is not piped to a command during a os.system call but it is during any of the subprocess commands because you are using stdout=PIPE.

If you piped the output from your test script to less for example you will find that subprocess method will also not work because you have now added a pipe to stdout of your test thus making ydiff.py have a pipe to less.

Ok boring part over: reading ydiff.py's option handling shows it has a --color option which it parses this way.

if (opts.color == 'always' or
    (opts.color == 'auto' and sys.stdout.isatty())):
    markup_to_pager(stream, opts)
else:
    # pipe out stream untouched to make sure it is still a patch
    byte_output = (sys.stdout.buffer if hasattr(sys.stdout, 'buffer')
               else sys.stdout)
    for line in stream:
        byte_output.write(line)

So you can add this to your command line to make it work.

if TEST_02:
    cmd_02_shell    = "diff -u f_01.txt f_02.txt | ydiff -s --color='always'"

Good question though as I like the output of ydiff.py

Small edit here:

Dont know if you realize but ydiff is an awesome for showing git diffs too. To try that out in your repo dir just do this. Accepts an optional file to diff.

$ ydiff -s [my_git_file]

Pretty cool if you ask me.

Mike Pennington · Accepted Answer · 2020-04-21 18:28:36Z

0

Running under os.system() seems to at least fix diff color rendering... but I haven't figured out why os.system() fixes it yet.

import os

TEST_04 = True
if TEST_04:
    os.system("diff -u f_01.txt f_02.txt | ydiff -s")

answered Apr 21, 2020 at 18:28

Mike Pennington

43.2k22 gold badges140 silver badges191 bronze badges

Collectives™ on Stack Overflow

color terminal side-by-side diffs broken in python

2 Answers 2

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Related