I have built a data pipeline in Python (3.12). It typically processes hundreds of thousands of files and runs for several days. A particular pipeline function calls external web APIs that sometimes fail. I expect some exceptions and I want to record them. However, if the exception frequency is above a threshold I do want to abort immediately.

I implemented an exception class that I use to decorate the function above.

Implementation:

class ExceptionMonitor:
    """ Monitor the number of exceptions raised and the number of calls made to a function. """
    def __init__(self, error_rate_threshold: float, min_calls: int):
        """
        :param error_rate_threshold: The error rate threshold above which an exception is raised.
        :param min_calls: The minimum number of calls before the error rate is calculated.
        """
        self.error_rate_threshold = error_rate_threshold
        self.min_calls = min_calls
        self.nb_exceptions = 0
        self.nb_calls = 0
        self.error_rate = 0

    def _save_exception(self):
        with open("exceptions.log", "a") as f:
            f.write(f"{datetime.now()}\n")
            f.write(traceback.format_exc())
            f.write("\n\n")


    def __call__(self, func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            try:
                return func(*args, **kwargs)
            except Exception:
                self.nb_exceptions += 1
                if self.nb_calls >= self.min_calls:
                    self.error_rate = self.nb_exceptions / self.nb_calls
                    if self.error_rate > self.error_rate_threshold:
                        raise
                self._save_exception()
            finally:
                self.nb_calls += 1
        return wrapper

Usage:

@ExceptionMonitor(error_rate_threshold=0.03, min_calls=100)
def some_func():
   call_web_api()

Is this code idiomatic?