I've been using the Python logging module for a couple of weeks now, and I want to like it because A) it's a standard module, B) it has some cool features like multiple handlers and hierarchy. But almost every time I use it I feel like I might as well just write my own logging module suitable for my purposes... because it seems like I have to do that anyways. The module just seems to require too much scaffolding and setup to use.
Here's what I mean. To do it properly you have to:
- get a logger
- set the verbosity level of the logger
- create a file or steam handler
- create a formatter (the default needs replacing)
- add the formatter to the handler
- add the handler to the logger
- do this all again if you want to mirror to stderr AND to a file (which is why I started using logging in the first place)
- put in code to shut down the logging (makes sure the streams get flushed) and for safety use the atexit module, meaning
- import atexit
- register the shutdown
- add an exception hook so that we can log uncaught exceptions too
This is just a little too much for basic proper use, don't you think?
To be fair, there is a "simple" way to use logging which is to just use the logging module functions "BasicConfig()" and "debug|info|warning|error|etc()" functions without getting a logger for your module. But it doesn't give the behaviour I want and even they prefer you don't use it in this manner.
What I believe is missing is a set of helper-functions and/or sytactic sugar to handle common tasks.
- let more things like Level and Handlers be put in the argument to getLogger
- automatically wrap common things like a File-like object and filename string into a handler instead of having the need to explicitly make one.
- an at-exit shutdown should be somewhat implicit (maybe an option to turn it off) as well as the option to trap other exceptions
And what I'd like to have for simple operation:
- one line (minus "import") to get a logger for my module with any optional formatting and whatnot.
- one line to configure the root logger with all options, that can deal with an array of logging destinations, that will auto-interpret formatting strings and destinations instead of needing to create sub-handlers and formatters, etc.
Here was my first crack at collapsing all that with two helper functions, but I hate having to add more functions to import for things that should have just been available (yes it's a bit ugly).
def add_to_logging(log,whereto=None,level=10,format="%(levelname)s: %(message)s",dateformat='%Y%m%d_%H%M%S'):
''' shortuct to attach a destination to an existing logging object
logfile can be file or gzip or stream or None(meaning stderr) '''
if whereto is None: whereto = sys.stderr
if isinstance(whereto,(str,unicode)):
fp = opener(whereto,'w')
else:
fp = whereto
fh = logging.StreamHandler(fp)
fh.setLevel(level)
if format:
formatter = logging.Formatter(format,dateformat)
fh.setFormatter(formatter)
log.addHandler(fh)
def setupLogging(logname=None,rootname='',timestamp=False,consoleLevel=20):
''' shortcut to set up a dual stderr/logname LOGGING stream
default level for file is DEBUG, for console is INFO
(set consoleLevel to 0 to turn off console)
SEE PYTHON LOGGING DOCUMENTATION FOR LOGGING BEHAVIOR
returns a logging object'''
import atexit
logger = logging.getLogger(rootname)
logger.setLevel(logging.DEBUG)
dateformat='%Y%m%d_%H%M%S'
# change the formatting if timestamp
fmtstring = "%(levelname)s: %(message)s"
if rootname is not None and rootname != '':
fmtstring = "%(name)s:" + fmtstring
if timestamp:
fmtstring = "[%(asctime)s:%(name)s:%(lineno)s:%(levelname)s] %(message)s"
# add a file if specified
if logname:
assert isinstance(logname,(str,unicode))
#logging.basicConfig(filename=logname,format=fmtstring,dateformat=dateformat)
add_to_logging(logger,logname,format=fmtstring,dateformat=dateformat)
# add a console
if consoleLevel!=0:
add_to_logging(logger,sys.stderr,format=fmtstring,level=consoleLevel,dateformat=dateformat)
# cleanup and exception handling
atexit.register(logging.shutdown)
# the following will capture exceptions to the logs as well
sys.excepthook = lambda *x: logger.error('Uncaught Exception',exc_info=x)
return(logger)
In the above "opener()" is a separate function I have that wraps opening a filename, file object, pipe, or what have you depending on the input and optionally with encoding. Sometimes I miss how easy that is dealt with in Perl.
