Skip to main content
Unix & Linux

Return to Answer

deleted 4 characters in body
Source Link
muru
muru
  • 77.9k
  • 16
  • 212
  • 318

There are several limitations that could be limiting the transfer speed.

1.) There is inherent network overhead on a 1Gbps pipe. Usually, this reduces ACTUAL throughput to 900Mbps or less. Then you have to remember that this is bidirectional traffic and you should expect significantly less than 900Mbps down.

2.) Even though you're using a "new-ish router" are you certain that the router supports 1Gbps? Not all new routers support 1Gbps. Also, unless it is an enterprise-grade router, you likely are going to lose additional transmit bandwidth to the router being inefficient. Though based on what I found below, it looks like you're getting above 100Mbps.

3.) There could be network congestion from other devices sharing your network. Have you tried using a directly attached cable as you said you were able to do?

4.) What amount of your disk IO are you using? Likely, you're being limited, not by the network, but by the disk drive. Most 7200rpm HDDs will only get around 40MB/s. Are you using raid at all? Are you using SSDs? What are you using on the remote end?

  1. There is inherent network overhead on a 1Gbps pipe. Usually, this reduces ACTUAL throughput to 900Mbps or less. Then you have to remember that this is bidirectional traffic and you should expect significantly less than 900Mbps down.

  2. Even though you're using a "new-ish router" are you certain that the router supports 1Gbps? Not all new routers support 1Gbps. Also, unless it is an enterprise-grade router, you likely are going to lose additional transmit bandwidth to the router being inefficient. Though based on what I found below, it looks like you're getting above 100Mbps.

  3. There could be network congestion from other devices sharing your network. Have you tried using a directly attached cable as you said you were able to do?

  4. What amount of your disk IO are you using? Likely, you're being limited, not by the network, but by the disk drive. Most 7200rpm HDDs will only get around 40MB/s. Are you using raid at all? Are you using SSDs? What are you using on the remote end?

I suggest using rsync if this is expected to be re-run for backups. You could also scp, ftp(s), or http using a downloader like filezilla on the other end as it will paralellizeparallelize ssh/http/https/ftp connections. This can increase the bandwidth as the other solutions are over a single pipe. A single pipe/thread is still limited by the fact that it is single-threaded, which means that it could even be CPU bound.

With rsync, you take out a large amount of the complexity of your solution as well as allows compression, permission preservation, and allow partial transfers. There are several other reasons, but it is generally the preferred backup method (or runs the backup systems) of large enterprises. Commvault actually uses rsync underneath their software as the delivery mechanism for backups.

Based on your given example of 80GB/h, you're getting around 177Mbps (22.2MB/s). I feel you could easily double this with rsync on a dedicated ethernet line between the two boxes as I've managed to get this in my own tests with rsync over gigabit.

There are several limitations that could be limiting the transfer speed.

1.) There is inherent network overhead on a 1Gbps pipe. Usually, this reduces ACTUAL throughput to 900Mbps or less. Then you have to remember that this is bidirectional traffic and you should expect significantly less than 900Mbps down.

2.) Even though you're using a "new-ish router" are you certain that the router supports 1Gbps? Not all new routers support 1Gbps. Also, unless it is an enterprise-grade router, you likely are going to lose additional transmit bandwidth to the router being inefficient. Though based on what I found below, it looks like you're getting above 100Mbps.

3.) There could be network congestion from other devices sharing your network. Have you tried using a directly attached cable as you said you were able to do?

4.) What amount of your disk IO are you using? Likely, you're being limited, not by the network, but by the disk drive. Most 7200rpm HDDs will only get around 40MB/s. Are you using raid at all? Are you using SSDs? What are you using on the remote end?

I suggest using rsync if this is expected to be re-run for backups. You could also scp, ftp(s), or http using a downloader like filezilla on the other end as it will paralellize ssh/http/https/ftp connections. This can increase the bandwidth as the other solutions are over a single pipe. A single pipe/thread is still limited by the fact that it is single-threaded, which means that it could even be CPU bound.

With rsync, you take out a large amount of the complexity of your solution as well as allows compression, permission preservation, and allow partial transfers. There are several other reasons, but it is generally the preferred backup method (or runs the backup systems) of large enterprises. Commvault actually uses rsync underneath their software as the delivery mechanism for backups.

Based on your given example of 80GB/h, you're getting around 177Mbps (22.2MB/s). I feel you could easily double this with rsync on a dedicated ethernet line between the two boxes as I've managed to get this in my own tests with rsync over gigabit.

There are several limitations that could be limiting the transfer speed.

  1. There is inherent network overhead on a 1Gbps pipe. Usually, this reduces ACTUAL throughput to 900Mbps or less. Then you have to remember that this is bidirectional traffic and you should expect significantly less than 900Mbps down.

  2. Even though you're using a "new-ish router" are you certain that the router supports 1Gbps? Not all new routers support 1Gbps. Also, unless it is an enterprise-grade router, you likely are going to lose additional transmit bandwidth to the router being inefficient. Though based on what I found below, it looks like you're getting above 100Mbps.

  3. There could be network congestion from other devices sharing your network. Have you tried using a directly attached cable as you said you were able to do?

  4. What amount of your disk IO are you using? Likely, you're being limited, not by the network, but by the disk drive. Most 7200rpm HDDs will only get around 40MB/s. Are you using raid at all? Are you using SSDs? What are you using on the remote end?

I suggest using rsync if this is expected to be re-run for backups. You could also scp, ftp(s), or http using a downloader like filezilla on the other end as it will parallelize ssh/http/https/ftp connections. This can increase the bandwidth as the other solutions are over a single pipe. A single pipe/thread is still limited by the fact that it is single-threaded, which means that it could even be CPU bound.

With rsync, you take out a large amount of the complexity of your solution as well as allows compression, permission preservation, and allow partial transfers. There are several other reasons, but it is generally the preferred backup method (or runs the backup systems) of large enterprises. Commvault actually uses rsync underneath their software as the delivery mechanism for backups.

Based on your given example of 80GB/h, you're getting around 177Mbps (22.2MB/s). I feel you could easily double this with rsync on a dedicated ethernet line between the two boxes as I've managed to get this in my own tests with rsync over gigabit.

Source Link
Khrystoph
Khrystoph
  • 459
  • 4
  • 4

There are several limitations that could be limiting the transfer speed.

1.) There is inherent network overhead on a 1Gbps pipe. Usually, this reduces ACTUAL throughput to 900Mbps or less. Then you have to remember that this is bidirectional traffic and you should expect significantly less than 900Mbps down.

2.) Even though you're using a "new-ish router" are you certain that the router supports 1Gbps? Not all new routers support 1Gbps. Also, unless it is an enterprise-grade router, you likely are going to lose additional transmit bandwidth to the router being inefficient. Though based on what I found below, it looks like you're getting above 100Mbps.

3.) There could be network congestion from other devices sharing your network. Have you tried using a directly attached cable as you said you were able to do?

4.) What amount of your disk IO are you using? Likely, you're being limited, not by the network, but by the disk drive. Most 7200rpm HDDs will only get around 40MB/s. Are you using raid at all? Are you using SSDs? What are you using on the remote end?

I suggest using rsync if this is expected to be re-run for backups. You could also scp, ftp(s), or http using a downloader like filezilla on the other end as it will paralellize ssh/http/https/ftp connections. This can increase the bandwidth as the other solutions are over a single pipe. A single pipe/thread is still limited by the fact that it is single-threaded, which means that it could even be CPU bound.

With rsync, you take out a large amount of the complexity of your solution as well as allows compression, permission preservation, and allow partial transfers. There are several other reasons, but it is generally the preferred backup method (or runs the backup systems) of large enterprises. Commvault actually uses rsync underneath their software as the delivery mechanism for backups.

Based on your given example of 80GB/h, you're getting around 177Mbps (22.2MB/s). I feel you could easily double this with rsync on a dedicated ethernet line between the two boxes as I've managed to get this in my own tests with rsync over gigabit.