Most files on a Linux system are normal files, i.e. they are saved on disk and reading from them just reads from a specified chunk of memory on the disk. How can I make something that behaves like a file in terms of being able to read from it as one would a normal file, but is actually returning programmatically generated data instead? As a concrete example, a file that downloads the current google.com and returns it, such that cat ~/myspecialfile would output the contents of google.com to stdout?
3 Answers
As the other answers have indicated, you can do part of what you've asked for using named pipes. For a complete solution, you'd have to develop some kind of virtual filesystem that took the desired actions when a path on the virtual filesystem was accessed. There are a few approaches to doing this:
- Write a kernel-mode filesystem driver, like the procfs driver.
- Write a user-mode filesystem implementation, using FUSE for example.
- Write a program which provides an NFS server interface or another network filesystem protocol.
- Maybe a program that pretends to be a USB file-storage device or another piece of hardware.
You can conceivably do something like this with a FIFO/named pipe:
$ mkfifo ~/myspecialfile
$ wget -q -O ~/myspecialfile google.com &
[2] 26186
$
mkfifo creates a named pipe called ~/myspecialfile. wget then directs its output to that named pipe. You can read (once) from that named pipe as if it were a regular file. e.g. to get wc counts for that file:
$ wc ~/myspecialfile
7 430 17738 /home/ubuntu/myspecialfile
[2]+ Done wget -q -O ~/myspecialfile google.com
$
Note that typically the writer (wget in this instance) will open() the pipe with the O_WRONLY flag. This open() call will typically block until the reader (wc in this instance) opens the other end of the pipe with the O_RDONLY flag. So here the wget process will be started immediately, but it will be doing nothing in the background until the reader (opens and) starts to read. I have verified this behaviour with wireshark on Ubuntu 14.04.
-
3This will run
wgetimmediately, though, right? Not just when the file is read?wchargin– wchargin2014-09-20 03:56:49 +00:00Commented Sep 20, 2014 at 3:56 -
@WChargin Yes, that's right, but it will sit in the background doing nothing until the pipe is read from. See my edit.Digital Trauma– Digital Trauma2014-09-22 17:56:57 +00:00Commented Sep 22, 2014 at 17:56
You can make myspecialfile a named pipe. Then you can have a script running in the background that's in a loop writing the output of the program to the pipe:
#!/bin/bash
mkfifo ~/myspecialfile
while :; do
run program here > ~/myspecialfile
done
There's a limitation to this. If two processes both open the file at the same time, they might each get only parts of the output.
A more elaborate way to do something that doesn't have this limitation would be to implement a FUSE filesystem. ~/myfyspecialfile would be the mount point of the filesystem.
/runthen, or else your software choices are going to become severely constrained. :-) (That's for Ubuntu, at least: on other flavors check/var/run,/var/tmpor similar locations.)