How to customize python logger behavior?

Question

I have implemented a custom_logger that my other python classes uses.

import logging, sys

logging.basicConfig(format='%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s',
    datefmt='%d-%m-%Y:%H:%M:%S',

logger = logging.getLogger()

I'm going to use it as follows:

import custom_logger as cl
cl.logger.debug('Extracting locations and descriptors from %s' % image_path)

I'm running my python files from a script in which I'm trying to redirect the logging in a different files for sout and serr

nohup python3  "$CLASSPATH""$MAIN_CLASS" $MAIN_CLASS_ARGUMENT 1>>$SOUT_FILE 2>>$SERR_FILE & echo $! > $PID_FILE

With the current setup, I get every log in the .err file

17-12-2018:17:56:59,203 INFO [conn.py:1008] Set configuration api_version=(0, 11, 0) to skip auto check_version requests on startup 17-12-2018:17:56:59,206 INFO [subscription_state.py:172] Updating subscribed topics to: ('PROCESSED_SCENES',) 17-12-2018:17:56:59,209 INFO [cluster.py:342] Group coordinator for my_group is BrokerMetadata(nodeId=0, host='172.1.1.3', port=9094, rack=None)

I tried tweaking the logger to output the error in each file, with no success:

import logging, sys

class StdErrFilter(logging.Filter):
    def filter(self, rec):
        return rec.levelno in (logging.ERROR, logging.WARNING)

class StdOutFilter(logging.Filter):
    def filter(self, rec):
        return rec.levelno in (logging.DEBUG, logging.INFO)

logging.basicConfig(format='%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s',
    datefmt='%d-%m-%Y:%H:%M:%S',
    level=logging.INFO)

formatter = logging.Formatter('%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s','%d-%m-%Y:%H:%M:%S')

logger = logging.getLogger()


h1 = logging.StreamHandler(sys.stdout)
h1.setFormatter(formatter)
h1.addFilter(StdOutFilter())
logger.addHandler(h1)

h2 = logging.StreamHandler(sys.stderr)
h2.setFormatter(formatter)
h2.addFilter(StdErrFilter())
logger.addHandler(h2)

All I'm getting now is having the same output in both .err .out files.

What am I missing?

dngrs · Accepted Answer · 2018-12-17 17:21:08Z

basicConfig configures a handler, so you have h1, h2 and basic logging. Maybe this causes confusion? Also you need to do logger.setLevel(logging.DEBUG) for the logger to process the debug level at all.

Removing basicConfig and adding setLevel seems to work as intended:

import logging, sys

class StdErrFilter(logging.Filter):
    def filter(self, rec):
        return rec.levelno in (logging.ERROR, logging.WARNING)

class StdOutFilter(logging.Filter):
    def filter(self, rec):
        return rec.levelno in (logging.DEBUG, logging.INFO)

formatter = logging.Formatter('%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s','%d-%m-%Y:%H:%M:%S')
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

h1 = logging.StreamHandler(sys.stdout)
h1.setFormatter(formatter)
h1.addFilter(StdOutFilter())
logger.addHandler(h1)

h2 = logging.StreamHandler(sys.stderr)
h2.setFormatter(formatter)
h2.addFilter(StdErrFilter())
logger.addHandler(h2)

logger.info('stdout log')
logger.error('stderr log')

Collectives™ on Stack Overflow

How to customize python logger behavior?

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related