The method of using find directory -print0 | xargs -0 should handle all specials. However, it requires one PID per file/directory, which can mount to a performance problem.

Let me describe another method of robust (and performant) file handling I have recently come across, which is suitable if find output should be post-processed as tab-separated CSV data, e.g. by AWK. In such processing, actually only tabs and newlines in file names are disruptive:

The directory is scanned via find directory -printf '%P\t///\n'. If the path does not contain tabs or newlines, this leads to one record with two CSV fields: the path itself and the field containing ///.

If a tab is contained in the path, there will be three fields: path fragment1, path fragment2 and the field containing ///.

If a newline is contained, there will be two records: first record will contain path fragment1 and the second record will contain path fragment2 and the field containing ///.

Now the key fact is that /// cannot naturally occur in paths. Also, it is a kind of waterproof escape or terminator.

It is also possible to write an (AWK) program that scans the find output and, until it finds ///, it puts the fragments together knowing that a new field is tab in the path and new record is newline in the path.

The tabs can be safely escaped as ///t and the newlines can be safely escaped as ///n, again, by knowing that /// cannot naturally occur in file paths. Converting ///t and ///n back to tabs and newlines can occur at the end, when some output is generated from the processing.

Yes, it sounds complicated, but the clue is that only two PIDs are needed: the find and the awk instance that performs the described algorithm. And it is fast.

The idea is not mine, I found it implemented in this new (2019) bash script for directory synchronization: Zaloha.sh. They have a docu there that describes the algorithm, actually.

I was not able to break/choke that program by special characters in filenames. It even correctly processed directories named newline and tab alone ...

JB

The method of using find directory -print0 | xargs -0 should handle all specials. However, it requires one PID per file/directory, which can mount to a performance problem.

Let me describe another method of robust (and performant) file handling I have recently come across, which is suitable if find output should be post-processed as tab-separated CSV data, e.g. by AWK. In such processing, actually only tabs and newlines in file names are disruptive:

The directory is scanned via find directory -printf '%P\t///\n'. If the path does not contain tabs or newlines, this leads to one record with two CSV fields: the path itself and the field containing ///.

If a tab is contained in the path, there will be three fields: path fragment1, path fragment2 and the field containing ///.

If a newline is contained, there will be two records: first record will contain path fragment1 and the second record will contain path fragment2 and the field containing ///.

Now the key fact is that /// cannot naturally occur in paths. Also, it is a kind of waterproof escape or terminator.

It is also possible to write an (AWK) program that scans the find output and, until it finds ///, it puts the fragments together knowing that a new field is tab in the path and new record is newline in the path.

The tabs can be safely escaped as ///t and the newlines can be safely escaped as ///n, again, by knowing that /// cannot naturally occur in file paths. Converting ///t and ///n back to tabs and newlines can occur at the end, when some output is generated from the processing.

Yes, it sounds complicated, but the clue is that only two PIDs are needed: the find and the awk instance that performs the described algorithm. And it is fast.

The idea is not mine, I found it implemented in this new (2019) bash script for directory synchronization: Zaloha.sh. They have a docu there that describes the algorithm, actually.

I was not able to break/choke that program by special characters in filenames. It even correctly processed directories named newline and tab alone ...