Wednesday, August 17, 2011

Log (logging) in Python

Logging is always an important topic when you try to develop a "real-world" app. I once wrote my own logging system, but apparently the best practice is to use something provided by Python Standard Library, not to reinvent the wheel.

Here, I have no intent to write a complete tutorial of teaching the python's logging framework. I extract the most useful/simple part(s) of logging from my viewpoints, which I believe it solves 90% cases I(you) need.


Firstly, you should try to build a customized logger. It is impractical to use the logging module directly via changing basicConfig() because you cannot tell where the log comes from if you have multiple modules in your app. This is one of the reason why I write this article.

To get a "customized" logger, firstly you should assign a name to it.
mlogging = logging.getLogger("test_log") #Get a logging for this module

If you would like to have some beautiful format on logging records, logging.Formatter is something you need to set it up. Sure you should do it because the default one seems to be a little clumsy. To change the date format (that is, %(asctime)s), please refer to things used in time.strftime(). And to create a format for a record, the related variables can be found in attributes of LogRecord.

# -- Set up Formatter --
log_format = '%(asctime)s(%(name)s--%(levelname)s):%(message)s'
log_date_format = '[%m/%d/%y %H:%M:%S]'
uniform_fmt = logging.Formatter(fmt=log_format, datefmt=log_date_format)

Mostly, we store logs in a file, so you need a file handler.
# -- Log to File --
hdlr = logging.FileHandler('OnlyOneFile.log')   # or "mlogger.name + 'log' if you would like to separate logs
hdlr.setFormatter(uniform_fmt)

If you would like to see log in your screen, say, for debugging, you have to link it to STDOUT.
# -- Log to Console --
hdlrConsole = logging.StreamHandler(sys.__stdout__)
hdlrConsole.setFormatter(uniform_fmt)

Do not forget to link your handler with your logger.
mlogging.addHandler(hdlr)
mlogging.addHandler(hdlrConsole)

Finally, you have to assign a log level so that you can get all the details. Otherwise, The default level is WARNING. (All default levels)
mlogging.setLevel(logging.DEBUG) #this is the most detailed level

Putting all together, here is an example:
import sys
import logging

mlogging = logging.getLogger("test_log") #Get a logging for this module
"""
logging.basicConfig(
    #filename=mlogging.name + '.log', #Use the name of logger as log file name
    filename='onefile.log', #Use the name of logger as log file name
    level=logging.DEBUG,
    format='%(asctime)s(%(name)s--%(levelname)s):%(message)s',
    datefmt='[%m/%d/%y %H:%M:%S]'
"""

# -- Set up Formatter --
log_format = '%(asctime)s(%(name)s--%(levelname)s):%(message)s'
log_date_format = '[%m/%d/%y %H:%M:%S]'
uniform_fmt = logging.Formatter(fmt=log_format, datefmt=log_date_format)

# -- Log to File --
hdlr = logging.FileHandler('OnlyOneFile.log')
hdlr.setFormatter(uniform_fmt)

# -- Log to Console --
hdlrConsole = logging.StreamHandler(sys.__stdout__)
hdlrConsole.setFormatter(uniform_fmt)

mlogging.addHandler(hdlr)
mlogging.addHandler(hdlrConsole)
mlogging.setLevel(logging.DEBUG)

mlogging.debug('This is debug msg')
mlogging.info('Ok~ info now')
mlogging.warning('Warning here')

try:
    x = 4/0
except Exception as e:
    mlogging.exception(e)

Before we leave the discussion, you should pay attention to mlogging.exception(e). This is the way we log an exception, including its trace-back information.