0

Is there a tool that allows http file download using multiple network interfaces and multiple connections? I'd like to test my overall internet connection speed by downloading a large file via multiple interfaces. Something like lftp with wget —bind-address option

Improve this question
9
  • Note that you can't download a single file (or web page) on multiple interfaces (unless the server you are contacting uses a multi-homing protocol like SCTP, but that's unlikely). You can download several files at the same time on different interfaces via different HTTP connections, though. Commented Nov 19, 2018 at 8:03
  • Why did you decide so? My client has multiple interfaces, what could prevent it to create several sockets bound to different addresses, connect them to the server and download different ‘range’ of file? Please explain Commented Nov 19, 2018 at 8:07
  • Because standard TCP and UDP protocols are single homed. Every connection must go through a specific interface, because you can only have a single IP address at each end. That's how the protocol works. So multiple interfaces won't make "your internet faster" (though many people wish this, this is a FAQ). The best you can do is to more or less randomly distribute connections among your interfaces. Or do a failover. Commented Nov 19, 2018 at 8:14
  • Of course you can write an application to download different parts of some file via HTTP on several connections, if the server supports it, but you must explicitly do so, and decide on the ranges, and I don't know any application that already does it. You could use bittorrent, which already makes use of many connections, but not HTTP. And bittorrent won't help with measuring speed. Commented Nov 19, 2018 at 8:16
  • ‘Every connection goes through a specific interface’ - correct. ‘Use of multiple interfaces won’t make overall speed faster’ - incorrect. Think again ))) Commented Nov 19, 2018 at 8:17

2 Answers 2

Reset to default
1

Yes - use wget with the --bind-address download option:

--bind-address=ADDRESS

When making client TCP/IP connections, bind to ADDRESS on the local machine. ADDRESS may be specified as a hostname or IP address. This option can be useful if your machine is bound to multiple IPs.

Simultaneously run a copy of wget on each interface. I am currently unable to test this, but I'm quite certain that the download will come from the interface that wget is bound to.

Alternatively, to allow multiple TCP connections on each network interface, you can use btdownloadcurses with the --bind option:

--bind ip

bind to ip instead of the default

Improve this answer
12
  • I know about this option, it works, but wget cannot download in multiple threads. This limits speed. Also, I couldn’t calculate the overall speed, and i need exactly this Commented Nov 17, 2018 at 19:44
  • I improved my question with this Commented Nov 17, 2018 at 19:45
  • 1
    I'm confused. If you run 2 wget processes at the same time, they will use different threads. Then, isn't the your overall speed just the sum of the speed of the individual downloads, e.g. 2.4 Mbps on one interface, 1.4 Mbps on another, overall 3.8 Mbps? What am I missing? Commented Nov 17, 2018 at 19:49
  • Not really. First, it,s hard to run them really simultaneously to check the overall speed. But it’s not the main issue. Second, single-thread download is usually slower, at least in my case (tried centos “all” image) Commented Nov 17, 2018 at 19:53
  • I updated my answer with an option that is multi-threaded per network interface. Perhaps a simple script could be used to run them simultaneously and even add the speeds together. Commented Nov 17, 2018 at 19:56
0

In case of HTTP or HTTPS, as long as server supports range requests you can fetch the ranges separately and stitch them together. I started working on a use case that is pointed by you. If you are still interested, here is a link to my repository https://github.com/m0hithreddy/MID.

  • The program (MID) uses SO_BINDTODEVICE socket option to bind to a specific interface, so in most of the cases you require super user permissions and CAP_NET_RAW capability (root user has).

  • MID determines the network interfaces to be used in the download and adopts two step split for downloading the content.

    • First step: The file is divided among network interfaces (in real time).
    • Second step: Further the file is divided among several HTTP range requests that arises from that particular interface (NOTE: Server should support them at the first place to make all of this possible)

Your use case ends here... But some more features of MID are... Support for HTTPS protocol, GZIP encodings (if server supports them).

If you are keen about using MID for HTTP only run ./configure with --without-ssl argument (./configure --without-ssl)

Cheers :)

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.