Extracting a partial part from filename using SED

Question

I am trying to understand this sed command to store something in a variable:

username=$(find . -iname '*.txt' | sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1./i' | sort - | uniq -ui |tr -d '\n')

I understand what sed does and the part at sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1./i' is basically taking out the regex equivalent of the username example SOMETHING_USERNAME.

find . -iname '*.txt' - Finds the name of the file that has the extension txt? and iname is used because it should ignore the case?

sort - sorts the files in order if there are more than one files?

uniq -ui allows to only store unique usernames.

tr -d deletes the rest?

I am trying to see if the understanding here is correct and if not how does it work.

adding the code for further help and more understanding for myself.

function process_zip {
    file="$1" #file is set to the INPUT
    folder="$file-$(date +%s)" #Setting Foldername

    declare -x   folder=${file%.*}     # Adding the file name to the left of the date and seconds.
    echo "filename to process" $file #printing filename


    echo "folderName" $folder #printing folder name
    mv "input/$file" in_progress #Move the folder from input to in_progress
    cd in_progress; #Go to progress

    # check file for validity before unzipping


    unzip -qq $file -d $folder; #not sure what -qq does exactly. This command unzips and checks if folder is available?
    echo "unzip completed" #prints
    cd $folder/placeholder/placeholder2; #goes into that folder?
    chmod -R 770 ** #Run recursively? understand this little but need more help.
    rsync -r * /placeholder1/placeholder2/placeholder3/placeholder4/; 
    echo "copy completed"
    #I want to use this next line so that the cut isn't hardcoded and works for files longer than 10 characters.
    #extract=$(find . -iname '*.txt' | sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1,/i' | sort - | uniq -ui | tr -d '\n')
    extract=$(cut -c -10 <<<"$file")
    echo "Extracted part is"$extract
    java -jar /placeholder1/placeholder2/placeholder3/placeholder4/placeholder5.jar $extract &
    cd ../../..; #back to in_progress
    pwd
    mv $file ../completed
    rm -r $folder &
    cd ../;
    echo "finished processing" $file
}

remaining=$(ls -1 input | grep .zip | wc -l) #It checks for more input files?

echo "${remaining} files to process"


while [ $remaining -gt 0 ]
do
    file=$(ls -t1 input| grep .zip | head -n1)
    echo "$file"
    process_zip "$file";

    remaining=$(ls -1 input | grep .zip | wc -l)
    echo "${remaining} files to process"
done;


find completed/* -mtime +15 -exec rm {} \;
find errors/* -mtime +15 -exec rm {} \;
find logs/* -mtime +15 -exec rm {} \;

echo "all done"

Thank you!

You can have a better title, read again about xed. Especially which have ton of examples. — Adi Prasetyo
– Adi Prasetyo, Commented May 31, 2022 at 0:58

Stéphane Chazelas · Accepted Answer · 2022-05-31 05:58:59Z

You have it pretty much correct. Here is how I would put it.

find . -iname '*.txt' Finds the names of files that have an extension txt, ignoring case (so ./wibble/wobble/wubble.Txt might be an example)

sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1./i' finds the last sequence of underscore followed by 4 digits followed by underscore followed optionally by a sequence of letters, digits, vertical bars and maybe some other characters in the file paths. If it finds such a sequence it throws everything else away, throws away the leading _ and appends a . character, otherwise it leaves the filename unchanged.

sort - sorts the filenames, respecting case (though the locale collation algorithm may ignore case in the first instance).

uniq -ui rejects names which appear more than once, ignoring differences in case.

tr -d '\n' joins all the filenames together by removing the newline characters.

This code looks to be fragile! It is probably expecting there to be a single file called something like sub/dir/pics_2023_happyxmas!/company/party/photos.txt and wants to extract 2023_happyxmas.. Adding another file with a txt extension will potentially give you another component to the resulting username variable, although you will be able to use . characters to split them apart.

What characters will be allowed to match might depend on the locale that the script is being run in.

Adding another file with an extension of txt but without the underscore digit digit digit digit underscore in the name will break the ability to use . to split apart the names.

If the program is being run in a controlled environment it could be OK, but I would certainly make the sed reject any lines it finds which do not match the expected pattern rather than passing them along unchanged.

Thanks for such a detailed explanation. I am trying to edit the code so that it takes in filenames such as 1230DAY_2020_VACATION and extracts 2020_VACATION and stores them into a variable to use that variable later in the code. I m new to bash so trying to understand as much as possible. Could you let me know how your suggestion would be implemented? And if this particular sed code is called in a function can it work with multiple files as well? — TigerStrips
– TigerStrips, Commented Jun 1, 2022 at 3:03
We can attempt to help. Can you provide some more information? Can there be more than one such filename? If so do you want to process them one at a time or all together? Would you consider the environment to be "friendly", so the filenames are not going to be created by people attacking your system? — icarus
– icarus, Commented Jun 1, 2022 at 4:41
I updated the post with more code. There are functions that are not being called but they are above this code to handle errors does bash run those functions with them being called? @icarus Can there be more than one such filename? If so do you want to process them one at a time or all together? -There can be more than one file but most likely not the same name. Would you consider the environment to be "friendly", so the filenames are not going to be created by people attacking your -- It is friendly — TigerStrips
– TigerStrips, Commented Jun 1, 2022 at 15:18

Stack Exchange Network

Extracting a partial part from filename using SED

1 Answer 1

You must log in to answer this question.

Linked

Hot Network Questions

Extracting a partial part from filename using SED

1 Answer 1

You must log in to answer this question.

Linked

Related

Hot Network Questions