How to know if the filesystem supports sparse file programmatically

Question

Java NIO provides us the option to create sparse files

    val path = Path("""E:\a""")

    val fileChannel = FileChannel.open(
        path,
        StandardOpenOption.READ,
        StandardOpenOption.WRITE,
        StandardOpenOption.CREATE_NEW,
        StandardOpenOption.SPARSE,
    )

The documentation says sparse file will be created if the OS and filesystem support it, but if it doesn't it will create a regular file.

But I want to know if the OS+filesystem does or does not support SPARSE so that, I can disable a feature from my app.

Answer 1

Sparse files and filesystems

According to this page, only 3 major filesystems don't support sparse files:

• HFS+ (old MacOS FS)
• exFAT (used for USB flash drives)
• FAT32 (very old Windows FS)

Also note that sparse files are not created the same way on different filesystems. On NTFS, you need to explicitly make the file sparse. On other filesystems, files are automatically sparse (as demonstrated in this question).

How Java creates sparse files

On Windows, the StandardOpenOption.SPARSE flag is used in WindowsChannelFactory:

// make the file sparse if needed
if (dwCreationDisposition == CREATE_NEW && flags.sparse) {
    try {
        DeviceIoControlSetSparse(handle);
    } catch (WindowsException x) {
        // ignore as sparse option is hint
    }
}

The call will succeed in case of NTFS.

For Unix-based OSes, the flag is ignored in UnixChannelFactory:

case SPARSE : /* ignore */ break;

The file will be sparse for most filesystems except HFS+.

Conclusion

There is no pure Java solution to answer the question "does the filesystem support sparse files". However, in practice, only USB flash drives and old MacOS may not.

Answer 2

If a file is a sparse file that means its physicalSize / logicalSize should be lover than 1. If you can calculate this values than you can undrestand if this file is sparse file or not.

To do that first you need to get the file is physicalSize. You should use different methods for windows and linux to calculate that value.

There is no direct API for that so you need to use ProcessBuilder for this:

for linux:

   Path path = Paths.get(filePath);
     File file = path.toFile();
    Long physicalSize = 0;
    // create and run the process
    Process process = new ProcessBuilder("du", "--block-size=1",file.getAbsolutePath()).redirectErrorStream(true).start();

        
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()))) {
            String line = reader.readLine();
            if (line != null) {
                String[] parts = line.split("\\s+");
                // take the value from return string  should be something like: 9379    filename
                physicalSize = Long.parseLong(parts[0]);
            }
        }

        process.waitFor();

After you capture the physical Size you need to capture the logical Size for the file, simply you can do it like this:

  Path path = Paths.get(filePath);
  long logicalSize = Files.size(path);

And finally you can use these 2 values to determine whether its a sparse file or not:

   double sparseness = (double) physicalSize / logicalSize;

            if (sparseness < 1) {
                System.out.println("a sparse file");
            } else {
                System.out.println("not a sparse file");
            }

Answer 3

You could create an empty file with size 10 KB, with sparse option, and check it's actual size on the filesystem. If it's actual size is less than 10 KB then sparse is supported. If it's 10KB - then it's not supported. And delete the file after the check.