0

we have a database that we take backup from it every night. backup files are 6 separate files in 6 separate directories. after backup is taken successfully, all parts will lftp to a remote server. the lftp commands are generated by a bash script which redirect output to a file named ftpfiles.sh then execute it. here is the content of ftpfiles.sh:

   lftp -u user,pass 1.1.1.1  <<end_script
   mkdir BackUp
   cd BackUp
   lcd /data10/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.001
   lcd /data1/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.002
   lcd /data2/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.003
   lcd /data7/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.006
   lcd /data8/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.005
   lcd /data9/customerBackup
   put CUSTOMER.0.DBPART000.20220705010003.004
   quit
   end_script
   exit 0

since each file is large, this process takes nearly 7 hours to be completed. so i wanted to send these files parallelly. if i want to use mirror option in lftp which can send files parallelly, i should add many includes and excludes which is risky. other option is to send each put process to background. here are the steps in command line (not using bash script):

  [root@autodb /]# lftp -u user,pass 1.1.1.1
  lftp [email protected].:~> mkdir BackUp
  lftp [email protected].:~> cd BackUp
  lftp [email protected].:~> lcd /data10/customerBackup
  lftp [email protected].:~> put CUSTOMER.0.DBPART000.20220705010003.001
  'CUSTOMER.0.DBPART000.20220705010003.001' at 40555088 (1%) [Sending data] {I PRESSED CTRL+Z IN HERE}
   [0] put CUSTOMER.0.DBPART000.20220705010003.001 &
    CUSTOMER.0.DBPART000.20220705010003.001 (8%) 45.85M/s eta:74s [Sending data]
  lftp [email protected].:~> quit
  [111] Moving to background to complete transfers...
  [root@autodb /]#

i tried to achieve this in bash script but no luck. for example putting these in bash script:

  (put CUSTOMER.0.DBPART000.20220705010003.001) &
   put CUSTOMER.0.DBPART000.20220705010003.001 &

so is it possible to use ctrl+z in bash script to send lftp process to background?
also here is the script that generates lftp commands:

  FTPFILES1='/data1/cronJobs/ftpfiles.sh'
  DBNAME=CUSTOMER
  ##ftp files
  cd /data1/cronJobs/
  cat /dev/null > ftpfiles.sh
  echo "lftp -u $FTPUSER,$FTPPASSWD $FTPSRV  <<end_script
  mkdir BackUp
  cd Backup " >> $FTPFILES1
  BACKUP_FILE_ARRAY=( /data*/"${DBNAME,,}"Backup/"$DBNAME.0.DBPART000".$(date +%Y%m%d)* )
     for BACKUP_FILE in "${BACKUP_FILE_ARRAY[@]}"
             do
             echo "lcd $(dirname $BACKUP_FILE)" >> $FTPFILES1
             echo "put $(basename $BACKUP_FILE)" >> $FTPFILES1
     done
  echo "quit
  end_script
  exit 0  " >> $FTPFILES1

1 Answer 1

2

You misunderstood about the bash. When you run the lftp command you are no longer in a bash but in the lftp environment and there is no CTRL+Z. In your case, I would suggest that you write 6 separate scripts. One for each file. and then write a BASH script that calls lftp individually. Then you can also use &. For example:

lftp -u user,pass 1.1.1.1  << script1 &
lftp -u user,pass 1.1.1.1  << script2 &
...
2
  • then i need to remove exit 0 Commented Jul 5, 2022 at 7:21
  • That depends on your use case. Maybe you have the chance to do some tests. Commented Jul 5, 2022 at 12:02

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.