How to check web page for changes from terminal?

Question

Is there a way to check if a web page changes using the command line? Something like this:

$ check-for-changes https://bbc.co.uk/web-page

https://bbc.co.uk/web-page just changed

Using a headless browser or some other mechanism?

Answer 1

First, get a fingerprint for the current state as a baseline:

curl --silent www.example.com | md5sum > www.example.com.md5

Then, you can subsequently check to see if the fingerprint changes:

curl --silent www.example.com | md5sum > www.example.com.md5new
if ! cmp www.example.com.md5 www.example.com.md5new > /dev/null; then
    printf "%s has changed from baseline!\n" "www.example.com"
fi
rm www.example.com.md5new

Answer 2

The site may serve pages with a LAST-MODIFIED header: do a HEAD request and extract that header.

curl --silent --head https://glennj.github.io | grep -i '^last-modified:'

Not all sites return that header though.