0

I would like to copy files from the current directory which are divided into different subdirectories. The files I want to copy should only have True and False as their parents. True directory files should be copied in dst/report/1/ and False directory files should be in dst/report/2/.

I haven't been able to figure out how to automate the copying process:

  • I thought I could use the find . -maxdepth digit but I don't know what the exact depth is.

  • The files should only be *.txt files, and even though some names may be duplicated, their contents are different, so I need either of them in the destination folder, but cp would overwrite them.

  • True and False intermediate parent directories could exist, but my intended files are generated in the last parent directory.

  • In addition, directory space is about 200GiB, so it takes a while to search through it.

  • I'm not sure about escape characters in directory and files so I don't really want to break the copying process.

sample working space:

```
__ current dir
  |__ path_1
        |__ True
              |__ 00000.txt
              |__ 020.txt
  |__ p_x
        |__ 100
              |__ True
                    |__ 00000.txt
                    |__ 020.txt
                    |__ 10.txt
              |__ False
                    |__ 1.txt
                    |__ 2.txt
                    |__ 200.txt
        |__ x
              |__ True
                    |__ 00000.txt
                    |__ 020.txt
              |__ False
                    |__ 1.txt
                    |__ 2.txt
        |__ True
              |__ path_2
                    |__ True
                        |__ 1.txt
  |__ x_p
        |__ ...
  |__ ...
  .
  .
  .

Expected results:

   __ dst
      |__ report
          |__ 1
              |__ 00000.txt
              |__ 020.txt
              |__ 00000_1.txt
              |__ 020_1.txt
              |__ 10.txt
              |__ 1.txt
  
          |__ 2
              |__ 1.txt
              |__ 2.txt
              |__ 200.txt
              |__ 1_1.txt
              |__ 2_2.txt

Update

As @muru answered cp could be handful but there is a constraint in number of argument it can handle through bash (getconf ARG_MAX).

shopt -s globstar
cp --backup=numbered **/True/*.txt dst/report/1/
cp --backup=numbered **/False/*.txt dst/report/2/

So back then, after executing these commands, was getting -bash: /bin/cp: Argument list too long in return.

Again as @muru mentioned in comments, it could be possible to solve this issue via xargs (assuming GNU xargs and cp).

printf "%s\0" **/True/*.txt | xargs -0 cp --backup=numbered -t dst/report/1/

In case of duplicate files with number extension (e.g; bar.txt.~1~, etc) rename would be useful.

prename 's/(.txt).~(\d+)~$/-$2$1/' dst/report/*/*, 

1 Answer 1

3

With bash, enable the recursive glob ** with shopt -s globstar and then use **/True/*.txt and **/False/*.txt as patterns, and if you have GNU cp, use the --backups or -b options to avoid overwriting files:

shopt -s globstar
cp --backup=numbered **/True/*.txt dst/report/1/
cp --backup=numbered **/False/*.txt dst/report/2/

Note that cp will add numbering after the extension:

% touch foo.txt
% cp foo.txt bar.txt                   
% cp --backup=numbered foo.txt bar.txt
% ls bar.txt*
bar.txt  bar.txt.~1~
% cp --backup=numbered foo.txt bar.txt
% ls bar.txt*                         
bar.txt  bar.txt.~1~  bar.txt.~2~
5
  • Thanks, Is there any way to change bar.txt.~1~ to bar-1.txt and so on for other files. Thinking of find . -type f -exec sed -i 's/.txt.~number~/-number.txt/g' {} + but not sure how to keep the number for later use in sed. Commented Jun 5, 2022 at 14:21
  • 1
    If you have the Perl rename available in your OS, you can try that: prename 's/(.txt).~(\d+)~$/-$2$1/' dst/report/*/*, but then you still risk some files getting overwritten (e.g., if you had x.txt, x-1.txt, x.txt.~1~). Commented Jun 5, 2022 at 15:11
  • Hmmm, thanks @muru . Also I made that to work like this find . -type f -not -name "*.txt" | rename -E 's/.txt.~([0-9]+)~/___$1.txt/g'. Commented Jun 5, 2022 at 15:43
  • Hmmm, seems to be not working, getting error bash: /usr/bin/cp: Argument list too long. Commented Jun 5, 2022 at 20:08
  • 2
    @LocalHosT if you get that, then you can use xargs to work around that: printf "%s\0" **/True/*.txt | xargs -0 cp --backup=numbered -t dst/report/1/ (assuming GNU xargs and cp) Commented Jun 6, 2022 at 1:10

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.