Is there a way via script to set an alert when an application is sending high volume of messages in short amount of time (5min)?
My goal here to have us notified when some application spamming or sending high amount messages.
Of course there is a way, however:
Although most monitoring systems provide monitoring of mail backlogs, I am not aware of any which monitor/alert based on rate
I believe you will need to write this yourself. It shouldn't be too complex, but you will to plan for how you deal with log rotation.
Out-of-the-box I don't believe that Postfix explicitly logs when an email is submitted (i.e. enters the incoming queue) only when a client connects (multiple messages can be sent in a single session) and when a message enters the active queue (may have multiple recipients). If you just look at the logs relating to delivery attempts you'll need to track back to find the From / client.
fail2ban
should do the job, it seems to have a postfic.conf filter.