Is it possible to automatically zip newly created files in a directory

Question

An application logs 100s of MB per hour.

I don't need historical logs and the logrotate utility doesn't run frequently enough. The application is closed source and rotates its own logs at around 36MB.

My Linux distribution is RHEL7.

Question

I'd like to compress and rotate the logs.

1. As the app splits out logs into new files, is it possible to automatically compress newly created files in a directory?
2. Is it possible to automatically delete all files in the format of assessor-cli.X.log where X is a digit greater than... say 5 (i.e. keep only 5 most recent logs).

Here is my logrotate file attempt:

# cat /etc/logrotate.d/cis_assessor
/usr/share/foreman-proxy/Ansible/CIS/audit/Assessor-CLI-4.0.2/logs/assessor-cli.log {
        missingok
        notifempty
        compress
        rotate 5
        size 30M

Logrotate jobs would need to catch the log between the size of 30MB and 36MB to actually come into effect which might only be a 10 second period. That's why I ask about the manual path of compressing and deleting files without logrotate.

Answer 1

As the app already splits out the logs into new files, is it possible to automatically compress newly created files in a directory?

Yes, it is. Just target the newly created file with something that can watch for new files in a directory (like entr)

So you'll create a logrotate config like this (/etc/logrotate.d/newlogrotateconf)

/usr/share/foreman-proxy/Ansible/CIS/audit/Assessor-CLI-4.0.2/logs/assessor-cli.log {
        missingok
        notifempty
        compress
        rotate 5
}

Then you'll run entr in a loop on the directory to tie logrotate into inotify/epoll,

echo -n /usr/share/foreman-proxy/Ansible/CIS/audit/Assessor-CLI-4.0.2/logs/
  | ./entr -dnc logrotate --force /etc/logrotate.d/newlogrotateconf