3

I am struggling with a small bash shell script that should list my csv files and upload them to a external Service using curl. when i echo my command variable out to console and execute it i works fine, but when i execute it within my script it complains about a extra qouta "

my script looks like this 

#!/usr/bin/bash
LOGDIR="/var/log/tyk/"
FILE_EXSTENSION_NAME="*.csv"
CURLCMD="curl -k -i -H"
PORT="9992"
ENDPOINT="/endpoint"
URL="https://localhost"



for i in `ls $LOGDIR$FILE_EXSTENSION_NAME`; do
    filename=`echo $i | awk -F "/" ' { print $5 }'`
    echo $filename
    CMD="$CURLCMD \"filename: $filename\" -F \"data=@$LOGDIR$filename\" $URL:$PORT$ENDPOINT"
    $CMD
done

When i run it i getfollowing output 

% Total    % Received % Xferd  Average Speed   Time    Time     Time     Current
                             Dload  Upload   Total   Spent    Left  Speed
 0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--0curl: (6) Could not resolve host: 2018-October-18-10.csv"; Unknown error
 0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--      0     
curl: (26) couldn't open file "/var/log/tyk/2018-October-18-10.csv""

If i do a echo $CMD in my script i got 

curl -k -i -H "filename: 2018-October-18-10.csv" -F "data=@/var/log/tyk/2018-October-18-10.csv" https://localhost:9992/endpoint

and that works ok

I cannot figure out what it is that i am not doing wrong

Improve this question
5
  • 1
    URL="https:localhost" - is that a typo? What is the exact error messasge you get? You show a curl error message that it can't open a file, but you talk about "missing quote" errors from bash. What is it? Commented Oct 18, 2018 at 11:38
  • yes it is URL=localhost" its because i changed it so i didnt expose the real URL. The errors is curl: (26) couldn't open file "/var/log/tyk/2018-October-18-10.csv"" there are two quotas in the end of the string but in my script i only set one. The file exists -rw-r--r--. 1 tyk tyk 1250 Oct 18 10:17 /var/log/tyk/2018-October-18-10.csv Commented Oct 18, 2018 at 11:48
  • 1
    Please edit your original question to correct the URL= part and also tell us the exact error message you get. Maybe instead of invoking awk, you want to use the basename tool to get the filename? Also, use ls -1 or better find to get at the list of filenames. And output $CMD before running it so you see what actually gets run. Commented Oct 18, 2018 at 11:51
  • there error is curl: (26) couldn't open file "/var/log/tyk/2018-October-18-10.csv"" Commented Oct 18, 2018 at 11:52
  • 2
    Don't put the command in a variable, just execute it directly (see BashFAQ #50: I'm trying to put a command in a variable, but the complex cases always fail!). Commented Oct 18, 2018 at 11:54

1 Answer 1

Reset to default
6

First of all, don't store commands in variables. Apart from that, there are two problems:

  1. Quotes inside expanded variables do not work as if you typed them in directly.
  2. The subshell $() around $($CMD) seems suspicous. Not only do you execute the command stored in $CMD, but you also execute the output of that command!

1. Quotes inside variables

When you enter the command echo "a b" then bash will process that command such that echo is executed with the argument a b. The output will be a b.

When you store that command inside a variable and expand that variable, bash will process the variable's content differently. The command echo will be executed with the arguments "a and b". The output will be "a b".

This is the reason for the error message

curl: (26) couldn't open file "/var/log/tyk/2018-October-18-10.csv""

curl is trying to open a path with an actual quote inside. Such a path does not exist on your system.

To get around this issue, you could write eval "$cmd", then the command would be executed as if you typed it in directly. However, you really shouldn't. I'd rewrite the script instead to not store commands in variables.

2. Subshell around $cmd:

cmd='echo something'
$cmd

This would print something. However, your script doesn't stop there, because you enclosed $cmd in $(). Therefore the output something gets executed too.

cmd='echo something'
$($cmd)

results in

bash: something: command not found
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks for the answer, i still gets the same result if i change $($CMD) to $CMD but if echo out my $CMD and run it manually it works fine
@havmaage The real reason was a bit hidden in my answer. I tried to make it more clear be reworking my answer.
Oh figured it out didn't read your explanation properly at the first time , ofcourse echo "a b" it makes sense changed my command to curl -k -i -H "filename: $filename" -F "data=@$LOGDIR$filename" $URL:$PORT$ENDPOINT and now it works thank you very much

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.