mosquitto bruteforce fail2ban failregex

Question

I am trying to write fail2ban filter to block bruteforcing IP addresses which are trying to subscribe to username/password protected mosquitto service by guessing username/password combinations. On attempt to subscribe with incorrect details, mosquitto writes two lines to log like:

1544984465: New connection from 123.123.123.123 on port 1883.
1544984465: Socket error on client <unknown>, disconnecting.

I managed to write regex matching this two lines pattern by using https://regex101.com/ set to "python flavor". Regex looks like this:

\s(?P<date>\d+)\: New connection from (?P<host>[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) .+\n(?P=date): Socket error on client \<unknown\>, disconnecting.

Unfortunately, fail2ban is unable to find matches in log by using this regex:

# fail2ban-regex '/var/log/testlog.log' '\s(?P<date>\d+)\: New connection from (?P<host>[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) .+\n(?P=date): Socket error on client \<unknown\>, disconnecting.'

Running tests
=============

Use   failregex line : \s(?P<date>\d+)\: New connection from (?P<host>[0-...
Use         log file : /var/log/testlog.log
Use         encoding : UTF-8


Results
=======

Failregex: 0 total

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [13] Epoch
`-

Lines: 13 lines, 0 ignored, 0 matched, 13 missed
[processed in 0.00 sec]

|- Missed line(s):
|  1544984465: New connection from 123.123.123.123 on port 1883.
|  1544984465: Socket error on client <unknown>, disconnecting.
|  1544984466: New connection from 123.123.123.123 on port 1883.
|  1544984466: Socket error on client <unknown>, disconnecting.
|  1544984468: New connection from 123.123.123.123 on port 1883.
|  1544984468: Socket error on client <unknown>, disconnecting.
|  1544984469: New connection from 123.123.123.123 on port 1883.
|  1544984469: Socket error on client <unknown>, disconnecting.
|  1544984470: New connection from 123.123.123.123 on port 1883.
|  1544984470: Socket error on client <unknown>, disconnecting.
|  1544984471: New connection from 123.123.123.123 on port 1883.
|  1544984471: Socket error on client <unknown>, disconnecting.
|  1544984473: New connection from 123.123.123.123 on port 1883.
`-

Fail2ban version is 0.9.6-1.el6.1. Running on Centos 6 server.

Answer 1

Ok, I made it. Problems were following:

Regular expressions (failregex, ignoreregex) assume that the date/time has been removed from the log line (this is just how fail2ban works internally ATM). Source: https://fail2ban.readthedocs.io/en/latest/filters.html

And I had to add maxlines = 2 to filter.

Final/working filter looks like this:

[Init]
maxlines = 2

[Definition]
failregex = .+ New connection from <HOST> on port \d+\.\n.+Socket error on client <unknown>
ignoreregex =