1

I am having a script which pulls data from different servers whose details it reads from external file. It reads the files and the verbose output shows all the matching files but it fetches only one file from the remote host. Following is my script:

while IFS=','; read region sdp ip1 ip2 ip3 user1 pass1 user2 pass2 user3 pass3
do

    in=/var/opt/fds/statistics/
    out=/pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    file=\*"PSC-TrafficHandler_8.1_A_"\*"_System."$date\*".stat"

    mkdir -p /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/

    ftp -in $ip1<<END_SCRIPT
    quote USER $user1
    quote PASS $pass1
    bin
    prompt off
    lcd /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    cd /var/opt/fds/statistics/
    binary
    mget *PSC-TrafficHandler_8.1_A_*_System.$date*.stat
    bye
    END_SCRIPT

done < /root/SDP_BHC/bin/Credentials.csv

Following is the output:

IP: 10.XXX.XX.XX

Interactive mode on.
Local directory now /pmautomation/PM/RawFiles/Data/BHCA/20150802/EAST/WB_SDP49
mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0000.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0100.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0200.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0300.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0400.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0500.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0600.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0700.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0800.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0900.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1000.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1100.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1200.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1300.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1400.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1500.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1600.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1700.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1800.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1900.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_2000.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_2100.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_2200.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_2300.stat?

Why my mget command is not able to get all the files and getting only one file from all the matching files?

Improve this question
4
  • I have defined the date variable above this code snippet. Commented Aug 3, 2015 at 12:24
  • 1
    it looks to me like your "prompt off" command is not having the desired effect; the mget output seems to be asking if if you want to retrieve each file. Commented Aug 3, 2015 at 14:55
  • The wildcards in the mget command may not be acting like you expect. Remember in linux "." is also a wildcard. Commented Aug 3, 2015 at 15:53
  • Thanks for the pointer Jeff. It was the same problem what you highlighted. I tried putting some 'y' under the mget and it fetched the files the number of time i put 'y'. This was really weird but it worked for me at least. Commented Aug 4, 2015 at 8:23

3 Answers 3

Reset to default
0

I have got some pointers from Jeff. Somehow the prompt off was not working and i was getting prompt to get the matching files. I have tried putting the 'y' below the mget command and it worked. Following is the updated code:

while IFS=','; read region sdp ip1 ip2 ip3 user1 pass1 user2 pass2 user3 pass3
do

    in=/var/opt/fds/statistics/
    out=/pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    file=\*"PSC-TrafficHandler_8.1_A_"\*"_System."$date\*".stat"

    mkdir -p /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/

    ftp -in $ip1<<END_SCRIPT
    quote USER $user1
    quote PASS $pass1
    bin
    prompt off
    lcd /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    cd /var/opt/fds/statistics/
    binary
    mget *PSC-TrafficHandler_8.1_A_*_System.$date*.stat
    y
    y
    y
    y
    bye
    END_SCRIPT

done < /root/SDP_BHC/bin/Credentials.csv

This is really strange behavior but it worked for me. Luckily i knew the number of files in remote host and putting the same number of 'y' made it work.

Improve this answer
0

Have you tried removing the tab from the end of the "here" document? Since the script is not detecting the end that may be your problem.

while IFS=','; read region sdp ip1 ip2 ip3 user1 pass1 user2 pass2 user3 pass3
do

    in=/var/opt/fds/statistics/
    out=/pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    file=\*"PSC-TrafficHandler_8.1_A_"\*"_System."$date\*".stat"

    mkdir -p /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/

    ftp -in $ip1<<END_SCRIPT
    quote USER $user1
    quote PASS $pass1
    bin
    prompt off
    lcd /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    cd /var/opt/fds/statistics/
    binary
    mget *PSC-TrafficHandler_8.1_A_*_System.$date*.stat
    bye
END_SCRIPT    <----------

done < /root/SDP_BHC/bin/Credentials.csv
Improve this answer
1
  • I think it should report error if it doesn't find the end. It was not reporting any error. Commented Aug 4, 2015 at 15:31
0

Use wget.

while IFS=','; read region sdp ip1 ip2 ip3 user1 pass1 user2 pass2 user3 pass3
do
    in=/var/opt/fds/statistics/
    out=/pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    file=\*"PSC-TrafficHandler_8.1_A_"\*"_System."$date\*".stat"

    mkdir -p $out
    cd $out

    wget --ftp-user=$user1 --ftp-password=$pass1 ftp://$ip1/$in/$file

done < /root/SDP_BHC/bin/Credentials.csv
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.