How can I extract part of the output of a curl command and assign it to a shell variable?

Question

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 277 100 174 100 103 464 274
--:--:-- --:--:-- --:--:--
736{"result":{"progressId":"ES_7PBXiq5gg67u9Vr","percentComplete":0.0,"status":"inProg
ress"},"meta":{"requestId":"**********************","httpStatus":"200 -
OK"}}

I need to extract the highlighted part beginning with ES_

I tried using

curl <...> | sed '"progressId"\:"(ES_[A-Za-z0-9]{15})'

however it still stores the entire output in the environment variable.

Answer 1

Best to use a json parsing tool to parse json. jq is a popular one.

set -o pipefail # if supported by your shell
var=$(curl -s "$url" | jq -r .result.progressId) || exit

Or:

json=$(curl -s "$url") || exit
var=$(printf '%s\n' "$json" | jq -r .result.progressId) || exit

(which allows you to extract more information from that json data later on).

Answer 2

This should work:

curl <...> | sed -E -n 's/.*(ES[^"]+).+/\1/p'

Explanation:

• .* means 'any char 0 or more times'
• ES[^"]+ means 'ES followed by any char which is not " one or more times'
• .+ means 'any char 1 or more times'
• \1 means 'the first group', so what's inside ()
• p means 'print only matching lines'

If you want to store it in a variable you can do it with backtick ` like this:

VAR="$(curl <...> | sed -E -n 's/.*(ES[^"]+).+/\1/p')"