Skip to main content
1 of 3
Marcus Müller
  • 51.5k
  • 4
  • 79
  • 121

What you're describing is multiplexing; something that requires a protocol (i.e., a formal specification on how to deal with data).

There's a lot of approaches to that. For example, you'll notice that your computer can perfectly well download multiple files at the same time via HTTP – even from the same server. That feature is foremost brought to you by TCP, which, as the transport protocol, allows different streams to be sent and "taken apart" at the receiving end.

So, TCP already offers that functionality, and you can simply start two concurrent SSH connections and make use of it!

zfs send | zstd -10 | ssh user@host sh -c 'zstd -d | zfs receive tank/vm@snapshot' &
# ^         ^    ^     ^             ^  ^  ^     ^   ^                             ^
# |         |    |     |             |  |  |     |   |                             |
# |         |    |     |             |  |  |     |   |     Tell your own shell to run this
# |         |    |     |             |  |  |     |   |     in the background and not block
# |         |    |     |             |  |  |     |   |                         
# |         |    |     |             |  |  |     |   |                            
# |         |    |     |             |  |  |     |   \---- Program to receive in the end
# |         |    |     |             |  |  |     |
# |         |    |     |             |  |  \-----+-------- use zstd to decompresse received
# |         |    |     |             |  |                  data
# |         |    |     |             |  |
# |         |    |     |             \--+----------------- tell the `sh` shell on the 
# |         |    |     |                                   server to run 'zstd -d | zfs…'
# |         |    |     |
# |         |    |     \---------------------------------- our first ssh invocaton
# |         |    |
# |         \----+---------------------------------------- use zstd to compress at a medium
# |                                                        high compression level (10)
# |
# \------------------------------------------------------- the first program whose output
#                                                          we send

cat somefile.txt | ssh user@host sh -c 'cat > somefile.txt'
#                                                          Second SSH connection

Of course, instead of the slightly inelegant cat somefile.txt | ssh … > somefile.txt, you would probably just use scp somefile.txt user@host:somefile.txt (which uses SSH under the hood, but doesn't do a shell connection, but uses the built-in SCP layer in SSH to copy the file).

You can make the connection establishment for the second connection faster by adding the following to your ~/.ssh/config file:

ControlMaster auto
IdentitiesOnly yes
ControlPath /tmp/.ssh-socket-%h_%p_%r

This will tell SSH to try and re-use the one SSH session to send multiple encrypted streams at the same time (this works with any combination of scp and ssh as well).

Marcus Müller
  • 51.5k
  • 4
  • 79
  • 121