Shell (.exp) script only sending 1 of 2 files

Question

I have a bash shell script that runs via cron. The script looks for 1 of 6 possible (data) files in a directory. If 1 or more of these files is found, a new (control) file is created for each data file found. The control file created here is associated (paired) to the data file found, by name, on a 1to1 basis. The main shell script works fine and calls the .exp script (shown below).

spawn sftp "$env(LOGINstring):/inbound/ach"
set timeout 7200
send "put $env(dataFILE)\n"
send "put $env(controlFILE)\n"
send "exit\n"
interact

Here is the data from the process log file.

Calling expect script to transmit the Welfare files...
spawn sftp [email protected]:/inbound/ach
Connected to remote-sftp-server.com.
Changing to: /inbound/ach
sftp> put NEI006AHB08659_WELF
Uploading NEI006AHB08659_WELF to /inbound/ach/NEI006AHB08659_WELF
NEI006AHB08659_WELF 0% 0 0.0KB/s --:-- ETA
NEI006AHB08659_WELF 100% 1710 79.6KB/s 00:00
sftp> put NEI007CTB08659_WELF
exit
Uploading NEI007CTB08659_WELF to /inbound/ach/NEI007CTB08659_WELF
Returned from expect script...

As you can see in the image, the "put" command for the data file (with 006AH) was executed and the data file was transmitted. Then the "put" command for the control file (with 007CT) was executed (or at least enough to display), but the .exp script's "exit" seems to have executed BEFORE the "put" command actually finished.

Thanks in advance!

Answer 1

I am not a expert in expect, but maybe you can use this code to send files to the desired destination:

scp "$dataFILE" "$LOGINstring":/inbound/ach
scp "$controlFILE" "$LOGINstring":/inbound/ach

You must use ssh keys to make the connection automatic (not sure how you deal with authentication in expect)

Answer 2

expect is not a programming language, like e.g. a shell or C, FORTRAN and the like. expect is more like a tool to automatically produce input / interpret output of an otherwise interactive session.

In your example that means: your expect script fires up a normal interactive sftp-session, just like you would do yourself. In this session commands ("put", "exit") are - "quasi-interactively" - entered and expect simulates these commands being entered from the keyboard.

Having said this: the script needs to behave like a user would and this means: wait for a command to complete. The basic structure of your program should look like:

spawn sftp
enter first "put" command
wait for the prompt to come back
enter second "put" command
wait for the prompt to come back
issue "exit"

The command to "wait" for a specific output is - "expect". See here for how to use that command.

Answer 3

From the expect(1) documentation, here is their example:

expect {
    Password: {
        stty -echo
        send_user "password (for $user) on $host: "
        expect_user -re "(.*)\n"
        send_user "\n"
        send "$expect_out(1,string)\r"
        stty echo
        exp_continue
    } incorrect {
        send_user "invalid password or account\n"
        exit
    } timeout {
        send_user "connection to $host timed out\n"
        exit
    } eof {
        send_user \
            "connection to host failed: $expect_out(buffer)"
        exit
    } -re $prompt
}

We can adapt your script with something such as:

spawn sftp "$env(LOGINstring):/inbound/ach"
set timeout 7200
send "put $env(dataFILE)\n"
expect "sftp>" { send "put $env(controlFILE)\n" }
expect "sftp>" { send "exit\n" }
interact

... to wait until the output buffer includes "sftp>"