Trying to design a system that can filter incoming email messages from a syslog server. Need to drop if they appear to be duplicates (throttle them). Procmail pipes them to a perl script for processing. That's all working, except if I get a flurry of emails all at once, postfix seems to spawn multiple instances of the piped script. Obviously this is desirable in most cases, but for this project, I have to be able to process the incoming messages one at a time so that a database field can be updated with a timestamp so that subsequent messages that match a criteria can be dropped instead of forwarded to the recipients. **Is there a way to "queue" these incoming messages to be processed one at a time, instead of in parallel? Thanks.
-
1Where do you want to do the queueing? You could set Postfix to only deliver one message at a time, or you could do a little magic in procmail. Which you'd choose depends on the results you are after. So, which will it be?user– user2017-05-26 15:59:45 +00:00Commented May 26, 2017 at 15:59
-
This is not high volume. I think I'll look into the procmail lock file idea first. Thanks.John Hawley– John Hawley2017-05-26 20:59:39 +00:00Commented May 26, 2017 at 20:59
Add a comment
|
2 Answers
You may use procmail's locking to make multiple instances execute one instance of the perl script at given time. See man procmailrc and man procmailex
:0 w: script.lock
| /.../script.pl
O course there are other ways but How many messages per hour do you expect to process in peak and average hour?
-
The last sentence of this answer really should have been a comment on the question. It does however not detract from the validity of the answer itself.user– user2017-05-26 21:02:51 +00:00Commented May 26, 2017 at 21:02
-
@MichaelKjörling I see it as a hint to someone wishing another solution.AnFi– AnFi2017-05-27 20:30:16 +00:00Commented May 27, 2017 at 20:30
To add to the solution by Andrzej, you can set a lock for a wider region than just a single recipe if you need to with the LOCKFILE special variable.
# Only one instance of Procmail can enter this region
LOCKFILE=.procmail-critical-region.lock
:0
* condition
| action
value=`program`
:0
* another condition
| another action
# We are done now; okay for another instance to enter
LOCKFILE=
In principle, you could have multiple critical sections which are guarded by the same lock file, though I am hard pressed to think of a scenario where this would actually be useful.
