Delete lines where text after character is duplicated

Question

I have a list of IP ranges in the following format:

Long description:111.22.33.0-111.22.33.40
Another description:5.5.5.0-5.5.5.100
Yet another description:111.22.33.0-111.22.33.40
And another one:111.22.33.0-111.22.33.40
Something different:8.1.1.0-8.1.1.20
etc.

I'd like to delete lines having duplicate IP ranges, even when the "description" differs (e.g. only match text after the : character).

The example above should become:

Long description:111.22.33.0-111.22.33.40
Another description:5.5.5.0-5.5.5.100
Something different:8.1.1.0-8.1.1.20

Answer 1

You could ask sort for the unique lines based on the 2nd field, delimited by colons:

sort -u -t: -k2 < input > output

The hard work is done by -u, which:

output(s) only the first of an equal run

(my emphasis)

Reference:

• https://linux.die.net/man/1/sort