Skip to main content
added 2041 characters in body
Source Link

adding the code for further help and more understanding for myself.

function process_zip {
    file="$1" #file is set to the INPUT
    folder="$file-$(date +%s)" #Setting Foldername

    declare -x   folder=${file%.*}     # Adding the file name to the left of the date and seconds.
    echo "filename to process" $file #printing filename


    echo "folderName" $folder #printing folder name
    mv "input/$file" in_progress #Move the folder from input to in_progress
    cd in_progress; #Go to progress

    # check file for validity before unzipping


    unzip -qq $file -d $folder; #not sure what -qq does exactly. This command unzips and checks if folder is available?
    echo "unzip completed" #prints
    cd $folder/placeholder/placeholder2; #goes into that folder?
    chmod -R 770 ** #Run recursively? understand this little but need more help.
    rsync -r * /placeholder1/placeholder2/placeholder3/placeholder4/; 
    echo "copy completed"
    #I want to use this next line so that the cut isn't hardcoded and works for files longer than 10 characters.
    #extract=$(find . -iname '*.txt' | sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1,/i' | sort - | uniq -ui | tr -d '\n')
    extract=$(cut -c -10 <<<"$file")
    echo "Extracted part is"$extract
    java -jar /placeholder1/placeholder2/placeholder3/placeholder4/placeholder5.jar $extract &
    cd ../../..; #back to in_progress
    pwd
    mv $file ../completed
    rm -r $folder &
    cd ../;
    echo "finished processing" $file
}

remaining=$(ls -1 input | grep .zip | wc -l) #It checks for more input files?

echo "${remaining} files to process"


while [ $remaining -gt 0 ]
do
    file=$(ls -t1 input| grep .zip | head -n1)
    echo "$file"
    process_zip "$file";

    remaining=$(ls -1 input | grep .zip | wc -l)
    echo "${remaining} files to process"
done;


find completed/* -mtime +15 -exec rm {} \;
find errors/* -mtime +15 -exec rm {} \;
find logs/* -mtime +15 -exec rm {} \;

echo "all done"

Thank you!

Thank you!

adding the code for further help and more understanding for myself.

function process_zip {
    file="$1" #file is set to the INPUT
    folder="$file-$(date +%s)" #Setting Foldername

    declare -x   folder=${file%.*}     # Adding the file name to the left of the date and seconds.
    echo "filename to process" $file #printing filename


    echo "folderName" $folder #printing folder name
    mv "input/$file" in_progress #Move the folder from input to in_progress
    cd in_progress; #Go to progress

    # check file for validity before unzipping


    unzip -qq $file -d $folder; #not sure what -qq does exactly. This command unzips and checks if folder is available?
    echo "unzip completed" #prints
    cd $folder/placeholder/placeholder2; #goes into that folder?
    chmod -R 770 ** #Run recursively? understand this little but need more help.
    rsync -r * /placeholder1/placeholder2/placeholder3/placeholder4/; 
    echo "copy completed"
    #I want to use this next line so that the cut isn't hardcoded and works for files longer than 10 characters.
    #extract=$(find . -iname '*.txt' | sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1,/i' | sort - | uniq -ui | tr -d '\n')
    extract=$(cut -c -10 <<<"$file")
    echo "Extracted part is"$extract
    java -jar /placeholder1/placeholder2/placeholder3/placeholder4/placeholder5.jar $extract &
    cd ../../..; #back to in_progress
    pwd
    mv $file ../completed
    rm -r $folder &
    cd ../;
    echo "finished processing" $file
}

remaining=$(ls -1 input | grep .zip | wc -l) #It checks for more input files?

echo "${remaining} files to process"


while [ $remaining -gt 0 ]
do
    file=$(ls -t1 input| grep .zip | head -n1)
    echo "$file"
    process_zip "$file";

    remaining=$(ls -1 input | grep .zip | wc -l)
    echo "${remaining} files to process"
done;


find completed/* -mtime +15 -exec rm {} \;
find errors/* -mtime +15 -exec rm {} \;
find logs/* -mtime +15 -exec rm {} \;

echo "all done"

Thank you!

edited title
Link

Help me understand sed command Extracting a partial part from filename using SED

added 202 characters in body
Source Link

I am trying to understand this sed command to store something in a variable:

username=$(find . -iname '*.txt' | sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1./i' | sort - | uniq -ui |tr -d '\n')

I understand what sed does and the part at sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1./i' is basically taking out the regex equivalent of the username example SOMETHING_USERNAME.

find . -iname '*.txt' - Finds the name of the file that has the extension txt? and iname is used because it should ignore the case?

sort - sorts the files in order if there are more than one files?

uniq -ui allows to only store unique usernames.

tr -d deletes the rest?

I am having a hard time understanding the rest of the statement. And if I wanttrying to just getsee if the string SOMETHING_USERNAME why do I need everything else?understanding here is correct and if not how does it work.

Thank you!

I am trying to understand this sed command to store something in a variable:

username=$(find . -iname '*.txt' | sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1./i' | sort - | uniq -ui |tr -d '\n')

I understand what sed does and the part at sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1./i' is basically taking out the regex equivalent of the username example SOMETHING_USERNAME. tr -d deletes the rest?

I am having a hard time understanding the rest of the statement. And if I want to just get the string SOMETHING_USERNAME why do I need everything else?

Thank you!

I am trying to understand this sed command to store something in a variable:

username=$(find . -iname '*.txt' | sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1./i' | sort - | uniq -ui |tr -d '\n')

I understand what sed does and the part at sed -e 's/.*_\([0-9]\{4\}_[0-9|A-z]*\).*/\1./i' is basically taking out the regex equivalent of the username example SOMETHING_USERNAME.

find . -iname '*.txt' - Finds the name of the file that has the extension txt? and iname is used because it should ignore the case?

sort - sorts the files in order if there are more than one files?

uniq -ui allows to only store unique usernames.

tr -d deletes the rest?

I am trying to see if the understanding here is correct and if not how does it work.

Thank you!

Source Link
Loading