grep partial ip number from the file

If I understand you correctly, you want to parse a list of IPs and identify which class B or C network they belong to. If any such network appears more than 10 times, you want to print the IP along with the network it belongs to in the notation

A.B.C.D   A.B.0.0/16  n

or

A.B.C.D   A.B.C.0/24  n

respectively into an output file spam.lst, where n is the actual occurence count of the respective subnet.

I propose the following awk program for the task (let's call it sort.awk):

#!/bin/awk -f

BEGIN{
    FS=OFS="."
}

NF==4{
    if (FNR==NR) {
        NF=cl
        count[$0]++
        next
    }
    for (n in count) {
        if (index($0,n)==1) {
            if (count[n]<=th) next
            printf "%s %s",$0,n
            for (i=cl;i<4;i++) printf ".0"
            printf "/%d %d\n",8*cl,count[n]
        }
    }
}

You would call it as follows:

awk -v cl=2 -v th=1 -f sort.awk ips.txt ips.txt> spam.lst

Note that the input file is processed two times, and hence appears two times as argument to awk!

The program works as follows:

• You specify the CIDR network class as awk variable cl as either 2 for a class B network, or 3 for a class C network.
• You specify the minimum occurence count from which on you want to block the entire subnet as awk variable th.
• The program sets the input and output separators to . to split input lines at the . into fields.
• The script only considers lines containing exactly 4 fields (minimum sanity check for IPs)
• In the first pass (FNR, the per-file line counter, is equal to NR, the global line counter), we register the subnets encountered. For every line, the field number is cut to the value in cl to truncate it to the class B or C network "base address". Then, a counter for this (newly regenerated) base address in the array count is increased, and processing skips to the next line.
• In the second pass, we iterate over all indices of count (i.e. all sub-nets registered in the first pass) to see if the IP on the current line starts with that subnet address. If the associated count is larger than the threshold, we output the current IP address, then the base address, padded to the right with .0 and with the netmask in CIDR notation appended, and finally the occurence count.

The output for cl=2, th=1 and the example IP list you showed would look like

108.61.115.213 108.61.0.0/16 2
108.61.199.100 108.61.0.0/16 2
138.68.224.206 138.68.0.0/16 2
138.68.235.36 138.68.0.0/16 2
148.66.129.250 148.66.0.0/16 2
148.66.130.114 148.66.0.0/16 2

The original proposition was meant to integrate into the existing script and looked as follows:

awk -v cl=2 -v nw="8.6.0.0" -F'.' 'BEGIN{split(nw,ref,/\./)} NF==4{for (i=1;i<=cl;i++) {if ($i!=ref[i]) next} printf "%s %s/%d\n",$0,nw,8*cl}' ips.txt

Here, we would parse the list of IPs to check whether they fall into the same network as a given network base address, specified via awk variable nw.

• In the beginning, the reference network base IP is split by fields into an array ref.
• For each line encountered, the program first checks if it contains 4 fields (minimum sanity check for an IP). If so, it compares the first cl fields of both the current line and the reference IP. If any of them doesn't match, the line is skipped and processing proceeds to the next line. If all relevant fields matched, the IP is printed, followed by the network in CIDR notation.