Python subprocess() reading file in bash

Question

I have a shell command for a file as given below:

filename="/4_illumina/gt_seq/gt_seq_proccessor/200804_MN01111_0025_A000H35TCJ/fastq_files/raw_data/200804_MN01111_0025_A000H35TCJ.demultiplex.log"

assembled_reads=$(cat $filename | grep -i " Assembled reads ...................:" | grep -v "Assembled reads file...............:")

Now I am trying to run this within a python environment using subprocess as:

task = subprocess.Popen("cat $filename | grep -i " Assembled reads ...................:" | grep -v "Assembled reads file...............:"", shell=True, stdout=subprocess.PIPE)
p_stdout = task.stdout.read()
print (p_stdout)

This is not working becasue I am not able to parse the filename variable from python to shell and probably there is a syntax error in the way I have written the grep command.

Any suggestions ?

Tried at the beginning of your first line of Python task = subprocess.Popen(f"cat ${filename} | ... (then the rest of that line) where you use that filename assignment you posted above in Python too? Any chance you can use IPython or Jupyter on your machine? You can use easier options of mixing in bash with Python in those cases. — Wayne
– Wayne, Commented Nov 28, 2021 at 20:08
The quotation marks are still unbalanced. You must escape the inner "s with a "\". — DYZ
– DYZ, Commented Nov 28, 2021 at 20:15
This works task = subprocess.Popen(f"cat {filename}", shell=True,stdout=subprocess.PIPE) — Bitsy
– Bitsy, Commented Nov 28, 2021 at 20:16

Tim Roberts · Accepted Answer · 2021-11-28 20:24:59Z

2

This code seems to solve your problem with no external tools required.

filename="/4_illumina/gt_seq/gt_seq_proccessor/200804_MN01111_0025_A000H35TCJ/fastq_files/raw_data/200804_MN01111_0025_A000H35TCJ.demultiplex.log"
for line in open(filename):
    if "Assembled reads" in line and "Assembled reads file" not in line:
        print(line.rstrip())

answered Nov 28, 2021 at 20:24

Tim Roberts

55.2k4 gold badges28 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Bitsy Over a year ago

Thanks, I decided to go for the pythonic way!

bubak · Accepted Answer · 2021-11-28 20:19:59Z

I would consider doing all the reading and searching in python and maybe rethink what you want to achieve, however:

In a shell:

$ export filename=/tmp/x-output.GOtV

In a Python (note the access to $filename and mixing quotes in the command, I also use custom grep command to simplify things a bit):

import os
import subprocess
tmp = subprocess.Popen(f"cat {os.environ['filename']} | grep -i 'x'", shell=True, stdout=subprocess.PIPE)
data = tmp.stdout.read()
print(data)

Though working, the solution is ... not what I consider a clean code.

Collectives™ on Stack Overflow

Python subprocess() reading file in bash

2 Answers 2

1 Comment

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Related