Skip to main content
Unix & Linux

Return to Question

Tweeted twitter.com/StackUnix/status/1118665564820574208
Fixed minor language issues
Source Link
terdon ♦
terdon ♦
  • 252.2k
  • 69
  • 480
  • 718

how to speed up tar,just just build a package without compression

iI have a biglarge folder, 2TB, with 1000000 files on a Linux machine. iI want to build a package with tar. iI do not care about the size of the tar file, so iI do not need to compress the data. howHow can iI speed tar up tar, it? It takes me aan hour to build a package with tar -cf xxx.tar xxx/tar -cf xxx.tar xxx/. iI have a powerful cpuCPU with 28 cores,and and 500GB memory，is there mutithreadinga way to speed up the tarmake tar run multithreaded?

or ifOr, alternatively, is there any good way to transfer hugea large number of small files between different folder andfolders and between different server. myservers? My filesystem is ext4. thanks for your help in advance

how to speed up tar,just build a package without compression

i have a big folder 2TB with 1000000 files. i want to build a package with tar. i do not care about the size of the tar file so i do not need compress the data. how can i speed up tar, it takes me a hour to build a package with tar -cf xxx.tar xxx/. i have a powerful cpu with 28 cores,and 500GB memory，is there mutithreading way to speed up the tar?

or if there any good way to transfer huge small files between different folder and between different server. my filesystem is ext4. thanks for your help in advance

how to speed up tar, just build a package without compression

I have a large folder, 2TB, with 1000000 files on a Linux machine. I want to build a package with tar. I do not care about the size of the tar file, so I do not need to compress the data. How can I speed tar up? It takes me an hour to build a package with tar -cf xxx.tar xxx/. I have a powerful CPU with 28 cores, and 500GB memory，is there a way to make tar run multithreaded?

Or, alternatively, is there any good way to transfer a large number of small files between different folders and between different servers? My filesystem is ext4.

Source Link
Guo Yong
Guo Yong
  • 141
  • 1
  • 1
  • 5

how to speed up tar,just build a package without compression

i have a big folder 2TB with 1000000 files. i want to build a package with tar. i do not care about the size of the tar file so i do not need compress the data. how can i speed up tar, it takes me a hour to build a package with tar -cf xxx.tar xxx/. i have a powerful cpu with 28 cores,and 500GB memory，is there mutithreading way to speed up the tar?

or if there any good way to transfer huge small files between different folder and between different server. my filesystem is ext4. thanks for your help in advance