I have a python subprocess call which I would like to link up to three pipes (two standard in and one standard out). I know that there is only one /dev/stdin, but there's all those other devices in /dev I don't know about, and don't know of any python os, sys or subprocess modules that will utilise them in a manner which allows me to give the device path to subprocess.Popen.
The reason I ask is because I would like to pipe information from a mysql database or tar archive rather than a directory structure I currently have which has >28,000 directories in. The directory names alone uses a LOT of space! The alternative is to tar / gunzip the entire directory structure and manoeuvre through the compressed archive. With either solution, mysql or tar, I would still like to have two pipes into subprocess.Popen and one out, so that I can bypass the HDD.
Any need for an example??
-
1Don't be afraid of punctuation. :)Lennart Regebro– Lennart Regebro2011-01-09 22:50:49 +00:00Commented Jan 9, 2011 at 22:50
Add a comment
|
1 Answer
On Unix systems, a convenient alternative is to use a named pipe. It looks like a file, but takes up no space on disk; you can write to it with one process and read from it with another, just like pipes. You can have your sub-process just do ordinary file I/O; Unix (Mac OS / Linux) will do the heavy lifting for you.
% mkfifo foo
% cat giantFile > foo &
[1] 4667
% wc -l foo
100
3 Comments
Alex Leach
cool, I've never used mkfifo or wc before.. Will have to look into them. Thanks :)
Alex Leach
I had a Eureka moment in bed last night after looking into the named pipes page you sent... ie. This is how to make all my code windows compatible! I've been relying on giving /dev/stdin & /dev/stdout as path names to subprocess calls all the time. This is the windoze fix! Awesome! Thanks again! If only the programs I call had windows executables / installers too....
Alex Leach
darn. python's os.mkfifo() is only on Unix OS's. However, found this link:- bytes.com/topic/python/answers/…