Python Logging

I've been using the Python logging module for a couple of weeks now, and I want to like it because A) it's a standard module, B) it has some cool features like multiple handlers and hierarchy.  But almost every time I use it I feel like I might as well just write my own logging module suitable for my purposes... because it seems like I have to do that anyways.  The module just seems to require too much scaffolding and setup to use.

Here's what I mean. To do it properly you have to:

• get a logger
• set the verbosity level of the logger
• create a file or steam handler
• create a formatter (the default needs replacing)
• add the formatter to the handler
• add the handler to the logger
• do this all again if you want to mirror to stderr AND to a file (which is why I started using logging in the first place)
• put in code to shut down the logging (makes sure the streams get flushed) and for safety use the atexit module, meaning
• import atexit
• register the shutdown
• add an exception hook so that we can log uncaught exceptions too

This is just a little too much for basic proper use, don't you think?

To be fair, there is a "simple" way to use logging which is to just use the logging module functions "BasicConfig()" and "debug|info|warning|error|etc()" functions without getting a logger for your module.  But it doesn't give the behaviour I want and even they prefer you don't use it in this manner.

What I believe is missing is a set of helper-functions and/or sytactic sugar to handle common tasks.

• let more things like Level and Handlers be put in the argument to getLogger
• automatically wrap common things like a File-like object and filename string into a handler instead of having the need to explicitly make one.
• an at-exit shutdown should be somewhat implicit (maybe an option to turn it off) as well as the option to trap other exceptions

And what I'd like to have for simple operation:

• one line (minus "import") to get a logger for my module with any optional formatting and whatnot.
• one line to configure the root logger with all options, that can deal with an array of logging destinations, that will auto-interpret formatting strings and destinations instead of needing to create sub-handlers and formatters, etc.

Here was my first crack at collapsing all that with two helper functions, but I hate having to add more functions to import for things that should have just been available (yes it's a bit ugly).

def add_to_logging(log,whereto=None,level=10,format="%(levelname)s: %(message)s",dateformat='%Y%m%d_%H%M%S'):
    ''' shortuct to attach a destination to an existing logging object
    logfile can be file or gzip or stream or None(meaning stderr) '''
    if whereto is None: whereto = sys.stderr
    if isinstance(whereto,(str,unicode)):
        fp = opener(whereto,'w')
    else:
        fp = whereto
    fh = logging.StreamHandler(fp)
    fh.setLevel(level)
    if format: 
        formatter = logging.Formatter(format,dateformat)
        fh.setFormatter(formatter)
    log.addHandler(fh)

def setupLogging(logname=None,rootname='',timestamp=False,consoleLevel=20):
    ''' shortcut to set up a dual stderr/logname LOGGING stream 
    default level for file is DEBUG, for console is INFO
    (set consoleLevel to 0 to turn off console)
    SEE PYTHON LOGGING DOCUMENTATION FOR LOGGING BEHAVIOR
    returns a logging object'''
    import atexit
    logger = logging.getLogger(rootname)
    logger.setLevel(logging.DEBUG)
    dateformat='%Y%m%d_%H%M%S'
    # change the formatting if timestamp
    fmtstring = "%(levelname)s: %(message)s"
    if rootname is not None and rootname != '':
        fmtstring = "%(name)s:" + fmtstring
    if timestamp:
        fmtstring = "[%(asctime)s:%(name)s:%(lineno)s:%(levelname)s] %(message)s"
    # add a file if specified
    if logname:
        assert isinstance(logname,(str,unicode))
        #logging.basicConfig(filename=logname,format=fmtstring,dateformat=dateformat)
        add_to_logging(logger,logname,format=fmtstring,dateformat=dateformat)
    # add a console
    if consoleLevel!=0:
        add_to_logging(logger,sys.stderr,format=fmtstring,level=consoleLevel,dateformat=dateformat)

    # cleanup and exception handling
    atexit.register(logging.shutdown)
    # the following will capture exceptions to the logs as well
    sys.excepthook = lambda *x: logger.error('Uncaught Exception',exc_info=x)
    return(logger)

In the above "opener()" is a separate function I have that wraps opening a filename, file object, pipe, or what have you depending on the input and optionally with encoding.  Sometimes I miss how easy that is dealt with in Perl.