change config file's format dynamically for python

Question

I have the following questions about python logging module.

Can I change the format defined in configuration file dynamically from code?
Can one module in my application get its format configured by a config file while other modules get their format and other properties configured through a python script?
I want to add contextual info to my logs but I am using a config file to define logger properties. How can I go about this?

P.S. I can provide some code and application structure if required.

Structure -

1.) Agent - calls logging_setup script to set up logger configuration 2.) Modules

I have a logging_setup script that has the following methods:

logging_setup.py

import logging
import logging.handlers

LOGGER_FORMAT="%(asctime)s %(CMDID)s %(levelname)s %(message)s"

class testFormatter(logging.Formatter):
  def format(self,record):
    record.message=record.getMessage()
    if string.find(self._fmt,"%(asctime)") >= 0:
      record.asctime = self.formatTime(record, self.datefmt)

    if threading.currentThread().getName() in cmdidDict:
      record.CMDID=cmdidDict[threading.currentThread().getName()]
    else:
      record.CMDID="Oda_EnvId"
    return self._fmt % record.__dict__

def initLogging(loggername,newCMDID):
  global CMDID
  global newCMDID
  logger=logger.getLogger(loggername)
  format=testFormatter(LOGGER_FORMAT)
  *set up other config*

So I wanted to set up CMDID variable in my logs dynamically which i achieved by using the testFormatter class and initializing logger with that class. There is a bit more to it but I hope u get the idea.

LOGGER_FORMAT="%(asctime)s %(CMDID)s %(levelname)s"

But now I want to change the log level of logs from the modules themselves keeping the format same throughout. So I decided to create a universal config file that could set up the logging configuration for me throughout the application. My problem is how do I make this config file set the CMDID variable in the format.

config file

[loggers]
keys=root,testModule

[formatters]
keys=generic

[handlers]
keys=fh

[logger_root]
level=DEBUG
handlers=fh

[logger_testModule]
level=ERROR        <<-- setting my log level here
handlers=fh
qualname=testModule
propagate=0

[handler_fh]
class=handlers.RotatingFileHandler
level=DEBUG
formatter=generic
maxBytes=1000
args=('spam.log',)

[formatter_generic]
format=%(asctime)s %(levelname)s %(message)s   <<---This should somehow set CMDID var

Try using ConfigParser and read various loglevels at runtime from the config section and insert it — Siva Cn
– Siva Cn, Commented Dec 12, 2013 at 19:39
Siva can you give and example of what do u mean by inserting it ?? Do u mean creating a new handler and adding it?? — tekken
– tekken, Commented Dec 13, 2013 at 7:05
Actually I just want that the format that is specified in config file should be overwritten by something that i define in my python code. How can I achieve that ?? — tekken
– tekken, Commented Dec 13, 2013 at 7:14

Siva Cn · Accepted Answer · 2013-12-16 04:24:44Z

You Could Try This:

import sys
import ConfigParser

CONFIG = ConfigParser.ConfigParser()

CONFIG.read('config.ini')  # This is your Config file

print "BEFORE CHANGE"
CONFIG.write(sys.stdout)

## Try printing any option under any section
#print CONFIG.get('section1', 'option1')

CONFIG.set('section1', 'option1', value=1234)

print "AFTER CHANGE"
CONFIG.write(sys.stdout)

OUTPUT:

BEFORE CHANGE
[section1]
option1 = aaaa
option2 = bbbb

[sectin2]
option1 = cccc
option2 = dddd

AFTER CHANGE
[section1]
option1 = 1234
option2 = bbbb

[sectin2]
option1 = cccc
option2 = dddd

JonathanV · Accepted Answer · 2013-12-12 17:54:51Z

I think providing that code and application structure may be needed.

It is not clear what you mean by changing the format. The format of what?
Sure. When you assign a value to variable_1 pull it from the config, then when you assign variable_2 get the value from somewhere else.
What contextual information do you want in your logs? You can configure your logs to have a field that is not predetermined and logs a variable value into them. So then all you have to do is set the value of that variable based on what is happening in your code and write it to the log.

Collectives™ on Stack Overflow

change config file's format dynamically for python

2 Answers 2

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Related