How to parse and add content to json using bash?

Question

I am following the ideas from the stackexchange articles below

• here to select and delete an element Delete Element
• here to append an element: Add New Element

but I am having problems with quotes when selecting and adding.

this is my sample json file

{
  "options": [
    {
      "label": "22",
      "value": "2022"
    },
    {
      "label": "23",
      "value": "2023"
    }
  ]
}

and this is my test bash script

#!/bin/bash

previous_year=$(date --date="$(date +'%Y') - 1 year" +%Y)
last_year=$((previous_year + 31))
label=$((last_year -2000))

echo ${last_year}

jq 'del(.options[]? | select(.label == \"${previous_year}\"))' temp.json
jq '.options += [{
      "label": '${label}',
      "value": '${last_year}'
    }]' temp.json

If I run the bash script with this sample file I get jq: error: syntax error, unexpected INVALID_CHARACTER (Unix shell quoting issues?) at , line 1: del(.options[]? | select(.label == "${previous_year}"))
jq: 1 compile error

If I remove the " it runs but don't find anything to delete because it looks without the quotes. It also add the element to the array but without quotes.

How can I do it?

• find the element with quotes
• add element with quotes

The expected result (what I want to achieve) is

{
  "options": [
    {
      "label": "23",
      "value": "2023"
    },
    {
      "label": "53",
      "value": "2053"
    }
  ]
}

Answer 1

Assuming you have the correct values in the shell variables label, last_year, and previous_year and that you want to delete all entries in the options array with the value value $previous_year, and add a new entry with the label value $label and value value $last_year.

jq  --arg add_label "$label" \
    --arg add_value "$last_year" \
    --arg del_value "$previous_year" '
    del(.options[]? | select(.value == $del_value)) |
    .options += [{ label: $add_label, value: $add_value }]' file

This correctly combines the two operations in a single invocation of jq.

The command instantiates three internal jq string variables with values taken from the shell variables that you have previously created. Using --arg to create internal string variables from shell variables ensures that the values are correctly encoded (this avoids code injection vulnerabilities).

As in your code, I use .options[]? rather than just .options[] to avoid an error if the options array does not exist. If it does exist, any entry with a value value of $del_value (an internal variable with the value from the shell variable previous_year) is deleted. You tried to select using the label, but I believe this might have been a typo.

The (possibly) modified document is passed on to the next stage, which adds a new element to the options array. If the array did not previously exist, this would create it with a single element.

Also note that jq does not do in-place editing, which means you will need to redirect the output of the above command to a new name and then replace the original file with that new file. Alternatively, use GNU sponge:

jq ...as above... file | sponge file

(Assuming your file is called file.)

Answer 2

#!/bin/bash
file=temp.json
now="$(date)"
previous_year=$(date --date="$(date +'%Y') - 1 year" +%Y)
last_year=$((previous_year + 31))
label=$((last_year -2000))
echo "==============================================================================================================="
echo "| Started parsing json file at script at: $file"
echo "| Started at:                             $now"
echo "| previous year is:                       $previous_year"
echo "| last year to be at json array is:       $last_year"
echo "| label assigned to the last year :       $label"
echo "==============================================================================================================="

if cat $file | grep $previous_year
then
  jq --arg key "$previous_year" 'del(.options[] | select(.value == $key))' $file >> new.json
  jq --arg key "$label" --arg val "$last_year" '.options += [{
        "label": $key,
        "value": $val
      }]' new.json >> new2.json
  mv new2.json $file
  rm new.json
  echo "last year found and parsed"  
else
  echo "nothing to be done"
fi

now="$(date)"
echo "==============================================================================================================="
echo "| Ended script at: $now"
echo "==============================================================================================================="
´´´