2

I want to use the "raster2pgsql" utility in my Python code. When I use it in a Linux terminal, it works fine. This is the command:

$ raster2pgsql -a "/mnt/c/Users/Jan/path/to/raster/dem.tiff" test_schema.raster2 | psql -h localhost -d pisl -U pisl

Then I use subprocess.run (I have also tried subprocess.call) to use this same tool in my Python code. This is my code:

from subprocess import run
command = ["raster2pgsql", "-a", '"' + file_name + '"', self.schema_name +  "." + identifier, "|", "psql", "-h", "localhost", "-p", "5432", "-d", self.dbname]
run(command)

I get this error:

ERROR: Unable to read raster file: "/mnt/c/Users/Jan/path/to/raster/dem.tiff"

Printing command gives this which I think is correct (equivalent to what worked in the terminal):

['raster2pgsql', '-a', '"/mnt/c/Users/Jan/path/to/raster/dem.tiff"', 'test_schema.raster2', '|', 'psql', '-h', 'localhost', '-p', '5432', '-d', 'pisl']

I have double checked that the path to the raster file is correct, tried single quotes, double quotes but nothing helps. I have looked at a number of similar question (here, here or here ) but did not find anything helpful.

I use Python 3.5 and Linux Bash Shell in Windows 10.

Question: What is wrong with the way I use subprocess?

Improve this question
3
  • 2
    why are you quoting the filename at all? Commented Jun 27, 2018 at 18:43
  • because it doesn't recognize the filename when I don't. When I remove the quotes it considers "test_schema.raster2" to be the filename. Commented Jun 27, 2018 at 18:45
  • I've provided an alternate method with 2 commands piped together. Please test that. report error accurately if any. I doubt that the quotes are the issue, since the shell removes them on the working command you performed. Commented Jun 27, 2018 at 18:48

1 Answer 1

Reset to default
5

2 issues here:

  • no need to extra-quote the filename. It's passed to the system literally and since there's no file called "/tmp/something" the command fails.
  • second, to be able to pass a pipe, you need shell=True

so quickfix:

command = ["raster2pgsql", "-a", file_name, self.schema_name +  "." + identifier, "|", "psql", "-h", "localhost", "-p", "5432", "-d", self.dbname]
run(command,shell=True)

or using a command string (because shell=True is picky with argument list):

command = "raster2pgsql -a "+ file_name + " " + self.schema_name +  "." + identifier + " | psql -h localhost -p 5432 -d" + self.dbname
run(command,shell=True)

(ugly, isn't it?)

It's much better to run 2 processes without shell=True and pipe them together using python, more portable & secure (not sure how shell=True reacts with an argument list on Linux):

from subprocess import *
command1 = ["raster2pgsql", "-a", file_name, self.schema_name +  "." + identifier]
p = Popen(command1,stdout=PIPE)
command2 = ["psql", "-h", "localhost", "-p", "5432", "-d", self.dbname]
run(command2,stdin=p.stdout)

The first Popen object created writes its output to a pipe (thanks to stdout=PIPE argument). The run function can take an input as a pipe too (thanks to stdin=p.stout). It consumes the output of the first command, creating a native piped chain of commands, without the need of the shell (and the caveats of quoting, spaces, special character interpretation and such...)

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thanks the for the explaination. I was trying to ask a py 3 script to run a py 2 script and I use process = subprocess.run(['py', '-2', os.path.basename(latest_file)]) but for some reason it didn't run. I accidentally came this and by adding shell=True in a different line, it works. Any chance you know why it works? Thank you so much!
you mean that subprocess.run(['py', '-2', os.path.basename(latest_file)],shell=True) runs and without shell=True it doesn't? that may well be another question to ask.
Oh quite the contrary. In one line it didnt' work. But in the format you proposed as a "dirty": command = ['py', '-2', os.path.basename(latest_file)] subprocess.run(command, shell=True)it works.
@Donald Li In my case if I use a Shell=True I have to use a string and not a list...

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.