bash delete line condition

Question

I couldn't find a solution to conditionally delete a line in a file using bash. The file contains year dates within strings and the corresponding line should be deleted only if the year is lower than a reference value.

The file looks like the following:

'zg_Amon_MPI-ESM-LR_historical_r1i1p1_196001-196912.nc' 'MD5'
'zg_Amon_MPI-ESM-LR_historical_r1i1p1_197001-197912.nc' 'MD5'
'zg_Amon_MPI-ESM-LR_historical_r1i1p1_198001-198912.nc' 'MD5'
'zg_Amon_MPI-ESM-LR_historical_r1i1p1_199001-199912.nc' 'MD5' 
'zg_Amon_MPI-ESM-LR_historical_r1i1p1_200001-200512.nc' 'MD5'

I want to get the year 1969 from line 1 and compare it to a reference (let's say 1980) and delete the whole line if the year is lower than the reference. This means in this case the code should remove the first two lines of the file.

I tried with sed and grep, but couldn't get it working.

Thanks in advance for any ideas.

Answer 1

You can use awk:

awk -F- '$4 > 198000 {print}' filename

This will output all the lines where the second date is later than 31/12/1979. This will not edit the file in-place, you would have to save the output to another file then move that in place of the original:

awk -F- '$4 > 198000 {print}' filename > tmp && mv tmp filename

Using sed (will edit in-place):

sed -i '/.*19[0-7][0-9]..\.nc/d' filename

This requires a little more thought, in that you will need to construct a regex to match any values which you don't want to be displayed.

Answer 2

Perhaps something like this:

awk -F- '{ if (substr($4,1,4) >= 1980) print }' input.txt