2

I'm creating new file for every line of text in a .txt file

file=/tmp/textFile.txt
while IFS= read -r line
do
        printf "%s\n" "$line" > /tmp/"$line"txt
done < "$file"

I always get an error saying file name too long, is there anyway to shorten the title for the new files to a certain number of characters?

7
  • What is the exact error and what outputs it? Unless the string in $line is several thousand characters long, there should be no actual issue in your script. It's unclear how long the lines in your input file are and what the maximum would be. Commented Dec 30, 2022 at 6:31
  • the lines are between 5000 and 30,000 long, the file names can even be sequencing numbers like 001 002 003 then I can rename them Commented Dec 30, 2022 at 6:36
  • "Title too long" is not a standard error message from any common Unix utility. How exactly and from where exactly do you get this error message? Commented Dec 30, 2022 at 6:46
  • 1
    If the lines don't contain slashes, there'll be issues at around 255 chars already, as that'd the limit for mist filesystems for a single filename (nor the full path): en.m.wikipedia.org/wiki/Comparison_of_file_systems If there are slashes, then creating the intermediste dirs would be the issue Commented Dec 30, 2022 at 8:28
  • 2
    Regarding the file names can even be sequencing numbers like 001 002 003 then I can rename them - so is split -l 1 all you need? Commented Dec 30, 2022 at 12:58

2 Answers 2

2

Usig awk to output the contents of the line to a filename with at most 100 characters (preceded by /tmp/, followed by .txt):

awk '{ name = "/tmp/" substr($0,1,100) ".txt"; print >name; close(name) }' /tmp/textFile.txt

If you need to do something about the / and . characters in each name, for example, change them to underscores (this avoids overwriting random files in the filesystem other than the ones in the /tmp directory):

awk '{ name = "/tmp/" substr($0,1,100) ".txt"; gsub("[/.]","_",name); print >name; close(name) }' /tmp/textFile.txt
2
  • textfilesplitter.com I found this site works amazing Commented Dec 30, 2022 at 10:47
  • 1
    @RyanHerter This seems irrelevant to the question the way it is posed at the moment. If you want to split the file into chunks of more than one line per file, and if the filenames done matter at all, then why don't you simply use the standard split command? Commented Dec 30, 2022 at 11:50
2

You can truncate the lines to e.g. 252 characters (which becomes 255 with txt added).

while IFS= read -r line
do
    printf "%s\n" "$line" > /tmp/"${line:0:252}"txt
done< /tmp/textFile.txt

This parameter expansion is from ksh93 and is also supported by Bash and newer versions of zsh, but not portable to sh. The native zsh syntax is $line[1,252] or ${line[1,252]}, the latter also supported by yash.

In any case, as, depending on the filesystem, the limit is often on the number of bytes rather than number of characters¹, the above may not work properly if the text has multibyte characters.


¹ macos can add further complications in that it may decompose characters such as those with diacritics into a decomposed form storing characters like é as U+0065 and U+0301 instead of just U+00E9 precomposed form.

2
  • Should it be ${line:0:1024}, IIRC that uses zero-based indexing. the limit for a single file name is 255 chars/bytes in many filesystems: en.m.wikipedia.org/wiki/Comparison_of_file_systems Commented Dec 30, 2022 at 8:26
  • @ikkachu Thanks for catching that. I'll check the limits on MacOS when I'm at my Mac again. Commented Dec 30, 2022 at 8:53

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.