There is https://github.com/i-rinat/longnamefs, which uses fuse. Fuse has a limit of 1024 octets in Linux <=6.14, and 4095 in 6.15.
Update: Since I am not very happy with any fuse solution, I developed an LD_PRELOAD hack specifically to deal with torrents. Works with transmission-daemon and rtorrent (and is not polished code - I only published it here because somebody upvoted the answer and I thought it might actually be of use for some hacker out there).
https://cvs.schmorp.de/enametoolong/
It compiles a shared object, which runs an external script (called "shortener"), as specified by an env variable:
LD_PRELOAD=/some/dir/enametoolong.so \
ENAMETOOLONG_SHORTENER=/some/dir/shortener \
transmission-daemon
It consists of a perl script, genwrap, that generates wrappers for common libc functions from a short prototype. E.g., for GNU/Linux on amd64, these and many more:
int openat (int, path, int, mode_t)
int open64 (path, int, mode_t)
int creat64 (path, mode_t)
int openat64 (int, path, int, mode_t)
int unlink (path)
int unlinkat (int, path, int)
int mkdir (path, mode_t)
int mkdirat (int, path, mode_t)
ssize_t readlink (path, char *, size_t)
ssize_t readlinkat (int, path, char *, size_t)
A generated wrapper looks like this:
static int (*orig_creat)(const char *, mode_t);
extern "C" int (creat) (const char * a0, mode_t a1)
{
if (toolong (a0))
a0 = shorten (a0);
if (!orig_creat)
orig_creat = (int (*)(const char *, mode_t))dlsym (RTLD_NEXT, "creat");
return orig_creat (a0, a1);
}
I use this in enametoolong.C, which, has a small tooshort function that checks if a filename is too long:
static bool
toolong (const char *path)
{
size_t clen = 0;
for (;;)
{
if (*path == '/')
clen = 0;
else if (!*path)
return false;
else if (++clen > NAMEMAX) // 255
return true;
++path;
}
}
And has a shorten function that is both unbelievably overdesigned and clumsy, and uses an external shortener script (written in Perl) to shorten such names and caches them in a std::unordered_map - transmission does a LOT, of file opens.
Having the shortener as an external program makes it easier to prototype shortening quickly in the language of choice. Here is an excerpt of the shortener I use:
for (split /\//, $path) {
if (255 < length) {
# remove short-enough extension
s/((?:\..{1,5}){0,3})\z//s;
my $ext = $1;
while (255 < length "$_$ext") {
# remove a single utf-8 char from the end
s/(?:[\x01-\x7f]|[\xc0-\xff][\x80-\xbf]+|.)\z//s;
}
$_ .= $ext;
}
push @short, $_;
}
my $short = join "/", @short;
The result does not generate unique filenames (obviously, it just shortens names, which might result in collisions, and the pigeonhole principle still applies), but it is very much faster and easier to use (well, for me) than a fuse filesystem that isn't a simple overlay. And the resulting torrent can be moved to a normal filesysytem easily.
It does fix my immediate need - download some torrents with long kanji filenames which are fine for windows, but too long for Linux.
I hope this is of some use, as a basis for your own shortener, maybe.
PS: I originally tried to move the dlsym calls into a constructor function, but the functions can be called long before the shared object constructors are called.
ext4limits are fine for you, you could use LUKS encryption, or the native encryption thatext4implements since recently. As forecryptfs, maybe you could improve it rather than introducing yet another new layer... I wish filesystems supported Unicode characters like NTFS rather than limiting to bytes in this day and age... you get the same problems whenrsyncfrom Japanese windows to ext4 because filenames can be longer in Windows when using non-ascii chars...ext4is not only one, I also usebtrfs.ecryptfsprovides clean mechanism to apply same encryption configuration for both. And as in question: I prefer overlay fs than LUKS. As you've mentioned, overlay FS for handling "too long filenames" could also address unicode issues as extra functionality.fuse-zipseems to handle files with long names and zips them (and I guess the zip would in turn be encrypted by ecryptfs). But it's probably not what you're looking for (and I don't know one that would do what Joliet/RockRidge is for CDs...)fuse-zipis not what I am looking for, but might work as "temporary workaround" for some directories/applications where I need more than 143 characters :). Thanks for this suggestion! And question remains open :).