Implicitly logging the filepath of the callsite of the log-statement

Question

Today I was challenged to build atop Python's logging module to automatically log the filepath from whence the log statement was issued.

The guidance in the https://docs.python.org/3/howto/logging.html#advanced-logging-tutorial is to logger = logging.getLogger(__name__) atop each module.

The question was: "Can I clean the callsite?"

So, for example:

# Foo.py
from LoggingHandler import log
class Foo:
    def __init__(self):
        log.debug('🔹Foo Debug')
        log.info('🔹Foo info')

# Bar.py
from LoggingHandler import log
class Bar:
    def __init__(self):
        log.debug('🔹Bar Debug')
        log.info('🔹Bar info')

# main.py
from Foo import Foo
from Bar import Bar
if __name__ == '__main__':
    foo = Foo()
    bar = Bar()

... would output:

2022-09-18 13:36:04,059-DEBUG-/path/to/Foo.py-🔹Foo Debug
2022-09-18 13:36:04,060-INFO-/path/to/Foo.py-🔹Foo info
2022-09-18 13:36:04,061-DEBUG-/path/to/Bar.py-🔹Bar Debug
2022-09-18 13:36:04,061-INFO-/path/to/Bar.py-🔹Bar info

This is what I came up with:

# LoggingHandler.py
import logging, inspect
from pathlib import Path

loggers = {}

def fetchOrCreateLogger():
    if (filepath := inspect.stack()[2].filename) not in loggers.keys():

        loggr = logging.getLogger(filepath)

        loggr.setLevel(logging.DEBUG)

        fh = logging.FileHandler(f'./{Path(filepath).stem}.txt')
        ch = logging.StreamHandler()

        fh.setLevel(logging.DEBUG)
        ch.setLevel(logging.DEBUG)

        formatter = logging.Formatter('%(asctime)s-%(levelname)s-%(name)s-%(message)s')
        fh.setFormatter(formatter)
        ch.setFormatter(formatter)

        loggr.addHandler(fh)
        loggr.addHandler(ch)

        loggers[filepath] = loggr

    return loggers[filepath]


class Bouncer:
    def __getattr__(self, item):
        return getattr(fetchOrCreateLogger(), item)

log = Bouncer()

I'm not sure whether I like it or not. I have little experience with this logging module. I would be grateful for insight from a better vantage point.

Kate · Accepted Answer · 2022-09-18 16:16:07Z

11

Unless I am missing something, adapting the formatter slightly would produce the same result:

formatter = logging.Formatter('%(asctime)s-%(levelname)s-%(pathname)s-%(message)s')

pathname (str) – The full string path of the source file where the logging call was made.

Source: logging — Logging facility for Python

answered Sep 18, 2022 at 16:16

Kate

8,3139 silver badges21 bronze badges

Add a comment |

janos · Accepted Answer · 2022-09-18 09:21:52Z

Inspecting the stack on every logger call

Although fetchOrCreateLogger caches the loggers to avoid creating them repeatedly, inspect.stack() is called for every logger method call (every log.debug, log.info, and so on). Note that inspecting the stack is usually not cheap.

You could replace log = Bouncer() with a factory method:

# in LoggingHandler
def callsite_path_logger():
    filepath = inspect.stack()[1].filename

    # ...

    return logger

And then use it in files that want this kind of logger as:

from LoggingHandler import callsite_path_logger


log = callsite_path_logger()

Notice that with this approach you don't need the cache anymore.

One caveat is that you must remember to not pass this logger to code in other files, because the logging calls will use filepath that was determined at construction time.

If you want a logger that will always report the correct path no matter from where it is called, then indeed you must inspect the call stack non every logging call, as in the original code. I don't recall such use case in practice, most of time loggers are created per file, using a factory method.

Transparent caching

The posted code manages the cache of loggers. You could benefit from functools.cache to do that for you, getting rid of the loggers cache:

@functools.cache
def logger_for(filepath):
    logger = logging.getLogger(filepath)

    # ...

    return logger


class Bouncer:
    def __getattr__(self, item):
        return getattr(logger_for(inspect.stack()[1].filename), item)

Minor issues

Instead of path in loggers.keys() you can write simply path in loggers.

Instead of this:

if (filepath := inspect.stack()[2].filename) not in loggers:

I think this is easier to read, with one statement per line:

filepath = inspect.stack()[2].filename
if filepath not in loggers:

I would rename loggr to the natural spelling logger.

Stack Exchange Network

Implicitly logging the filepath of the callsite of the log-statement

2 Answers 2

Inspecting the stack on every logger call

Transparent caching

Minor issues

You must log in to answer this question.

Hot Network Questions

Implicitly logging the filepath of the callsite of the log-statement

2 Answers 2

Inspecting the stack on every logger call

Transparent caching

Minor issues

You must log in to answer this question.

Related

Hot Network Questions