Parse two-dimensional JSON arrays in bash

Question

I am currently somehow frustated because I confused how to parse JSON in bash.

I do a basic cURL request to a restful API in the bash script and get a JSON array as response, which is looking like this with multiple entries:

[{"id":"012.mp4","language":"en","timestamp":1407517051,"name":"012.mp4"},{"id":"013.mp4","language":"en","timestamp":1407512051,"name":"013.mp4"}]

Now I have to parse this JSON array in my Bash script. My goal is to loop over all entries and (w)get the ressource specified with "name". So in this example above I would like to

wget 012.mp4

wget 013.mp4

I would appreciate any tips or snippets how to archive that. I don't even have any clue how to do it, I read something about using python but I have no idea of python.

Here's the library for this: github.com/dominictarr/JSON.sh — should let you start towards your aim. — JimiDini
– JimiDini, Commented Nov 13, 2014 at 17:21
The reason that using python (or ruby or any other scripting language with packages for that matter) is a good idea is that they have packages that provide JSON parsing and give you higher level data structures to work with. I highly encourage you to use a different language than bash to do this, the effort will probably be less in the end than figuring out how to do it correctly with bash. — Max Leske
– Max Leske, Commented Nov 13, 2014 at 17:24
This doesn't look like a two-dimensional array to me at all -- it's a one-dimensional array with objects (dictionaries/hashes) inside. — Charles Duffy
– Charles Duffy, Commented Nov 13, 2014 at 17:29
in bash what would you want to do that to yourself? Using any higher level scripting language would make this so easy and clear and readable. — njzk2
– njzk2, Commented Nov 13, 2014 at 17:33
@njzk2, is the solution I gave none of those things? Bash would be unsuited if you wanted to work with arbitrary data structures (as its dictionaries and arrays don't nest without some severe hackery I'm loathe to go into), but that isn't the case here. — Charles Duffy
– Charles Duffy, Commented Nov 13, 2014 at 17:35

Charles Duffy · Accepted Answer · 2014-11-13 17:31:02Z

4

The jq tool provides an easy-to-use JSON parser for bash:

while read -r name; do
  wget "$name"
done < <(curl http://your.url/ | jq -r .[].name)

You can test this directly:

jq -r .[].name <<<'[{"name": "hello"}, {"name": "world"}]'

returns:

hello
world

answered Nov 13, 2014 at 17:31

Charles Duffy

299k43 gold badges440 silver badges495 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

njzk2 Over a year ago

that actually does not look half bad!

akostadinov · Accepted Answer · 2014-11-13 17:29:21Z

0

If you don't want full blown json parsing, for such simple file you can also hack something quick and dirty like:

sed -En -e 's/}/\n/' -e 's/.*name":"(.*mp4)".*/\1/p' < file.json

Also you can try json_reformat command and then pass to sed for extracting the values.

answered Nov 13, 2014 at 17:29

akostadinov

19k6 gold badges86 silver badges94 bronze badges

Comments

ajaaskel · Accepted Answer · 2014-11-13 21:33:01Z

0

Pure BASH --- just for fun:

#!/bin/bash

string='[{"id":"012.mp4","language":"en","timestamp":1407517051,"name":"012.mp4"},{"id":"013.mp4","language":"en","timestamp":1407512051,"name":"013.mp4"}]'

string2="${string//\},\{/ }"
oldifs="$IFS"; IFS=' '
array=( $string2 )
IFS="$oldifs"

for i in "${array[@]}"; do
after_last_colon="${i##*:}"
echo "wget ${after_last_colon//[\}\]]/}"
done <<< "$array"

Result:

wget "012.mp4"
wget "013.mp4"

answered Nov 13, 2014 at 21:33

ajaaskel

1,73912 silver badges13 bronze badges

Comments

Stephen Quan · Accepted Answer · 2019-10-27 23:15:50Z

I created my own jq using python. With python we can import the json content into a dictionary. Then, we use python syntax to access any portion of the dictionary.

data - pretty print the JSON
len( data ) - 2
data[ 0 ][ 'name' ] - 012.mp4
data[ 1 ][ 'name' ] - 013.mp4

Here's a working demo:

sample() {
  cat <<EOF
[{"id":"012.mp4","language":"en","timestamp":1407517051,"name":"012.mp4"},{"id":"013.mp4","language":"en","timestamp":1407512051,"name":"013.mp4"}]
EOF
}

jq_py() {
cat <<EOF
import json, sys
data = json.load( sys.stdin )
print( json.dumps( $1, indent = 4 ) )
EOF
}

jq() {
  python -c "$( jq_py "$1" )"
}

unquote_py() {
cat <<EOF
import json,sys
print( json.load( sys.stdin ) )
EOF
}

unquote() {
  python -c "$( unquote_py )"
}

sample | jq "data"
# [
#     {
#         "timestamp": 1407517051,
#         "id": "012.mp4",
#         "language": "en",
#         "name": "012.mp4"
#     },
#     {
#         "timestamp": 1407512051,
#         "id": "013.mp4",
#         "language": "en",
#         "name": "013.mp4"
#     }
# ]

sample | jq "len( data )"
# 2

sample | jq "data[ 0 ][ 'name' ]" | unquote
# 012.mp4

sample | jq "data[ 1 ][ 'name' ]" | unquote
# 013.mp4

Collectives™ on Stack Overflow

Parse two-dimensional JSON arrays in bash

4 Answers 4

1 Comment

Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

Comments

Comments

Related