4.9. WBEM operation logging

New in pywbem 0.11 and redesigned in pywbem 0.12. Finalized in pywbem 0.13.

4.9.1. Pywbem logging overview

The pywbem package implements selected logging using the Python logging facility. This section describes logging for use of the pywbem package as a WBEM client. Section Logging in the listener describes logging for use of the pywbem package as a WBEM listener.

Pywbem logging is used to record information passing between the pywbem client and WBEM servers but not as a general recorder for errors, state, etc. within pywbem. In effect it is a trace tool. Pywbem errors are generally passed to the pywbem API user as Python exceptions rather than being recorded in a log by a pywbem logger.

Pywbem logging uses two Python Logger objects which are termed pywbem loggers:

  • API logger (Python logger name: ‘pywbem.api’) - Logs API calls and returns, for the WBEMConnection methods that drive WBEM operations (see WBEM operations). This logs the API parameters and results, including CIM objects/exceptions. It also logs the creation of WBEMConnection objects to capture connection information in order to determine the connection to which a particular log record belongs.

  • HTTP logger (Python logger name: ‘pywbem.http’) - Logs HTTP requests and responses between the pywbem client and WBEM server. This logs the HTTP request data and response data including HTTP headers and CIM-XML payload.

Pywbem uses the DEBUG logging level (see Logging Levels) for both loggers.

Pywbem adds a null handler to the logger named ‘pywbem’, in order to prevent the “No handlers could be found for logger …” warning. This follows best practices recommended in Configuring logging for a library and in several articles, for example in this article. Because this warning is no longer issued on Python 3.5 and higher, pywbem adds a null handler only on Python 2.7.

Because pywbem logs only at the DEBUG logging level (see Logging Levels), these log events will not be printed by the Python root logger by default, and therefore it is not necessary that pywbem attaches a null handler to any of its loggers.

There are two steps to setting up pywbem logging:

  • Configure Python logging parameters

    Sets the Python logging parameters for a pywbem logger or its parent loggers, such as log handler, message format, and logging level.

    This can be done with Python logging functions or with the functions described in Logging configuration functions.

  • Activate WBEM connection(s) for logging and set detail level

    In order to save the cycles for capturing the possibly large amounts of data needed for creating the log records, pywbem logging is inactive by default. Logging can be activated for an existing WBEM connection, or for all subsequently created WBEM connections.

    Because the pywbem loggers can generate large quantities of information, the user can control the quantity of information produced by each pywbem logger by setting a detail level for each logger when activating a WBEM connection for logging.

    Activation and setting detail levels are pywbem features so it requires using the functions described in Logging configuration functions.

4.9.2. Logging configuration functions

The pywbem loggers may be configured and/or WBEM connections may be activated for logging through the following pywbem functions.

These functions are the only mechanism for setting the detail level of a pywbem logger and for activating WBEM connection(s) for logging.

  • configure_logger() - Configure the pywbem loggers and optionally activate WBEM connections for logging and setting a log detail level.

  • configure_loggers_from_string() - Configure the pywbem loggers and optionally activate WBEM connections for logging and setting a log detail level, from a log configuration string.

    This is most useful in defining the pywbem logging from a command line tool such as pywbemcli so the complete logger configuration can be compressed into a single command line string.

4.9.3. Logging configuration examples

Examples for using configure_logger() for configuring pywbem loggers and for activating WBEM connections for logging:

  • Example: Configure the ‘pywbem.api’ logger with detail level ‘summary’ and output to stderr, and activate all subsequently created WBEM connections for logging:

    configure_logger('api', log_dest='stderr', detail_level='summary',
                     connection=True)
    
    # All of the following connections will log to stderr with summary output:
    conn1 = WBEMConnection(...)
    conn2 = WBEMConnection(...)
    
  • Example: Configure all pywbem loggers with the default detail level (‘all’) and output to a file, and activate a single existing WBEM connection for logging:

    conn = WBEMConnection(...)
    
    configure_logger('all', log_dest='file', log_filname='my_logfile.log',
                     connection=conn)
    

Examples for configuring the pywbem loggers using Python logging methods, and using the pywbem logging configuration functions only for setting the detail level and for activating WBEM connections for logging:

  • Example: Configure the pywbem parent logger (named ‘pywbem’) for logging to a rotating file, configure both pywbem loggers for detail level ‘summary’, and activate all subsequent WBEM connections for logging:

    import logging
    from logging.handlers import RotatingFileHandler
    
    logger = logging.getLogger('pywbem')
    handler = RotatingFileHandler("my_log.log", maxBytes=2000, backupCount=10)
    handler.setLevel(logging.DEBUG)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(message)s')
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    
    # configure without setting log_dest
    configure_logger('api', detail_level='summary', connection=True)
    
    # All of the following connections will log using the rotating file handler:
    conn1 = WBEMConnection(...)
    conn2 = WBEMConnection(...)
    

Examples for using configure_loggers_from_string() for configuring the pywbem loggers and for activating WBEM connections for logging:

  • Example: Configure the ‘pywbem.api’ logger with detail level ‘summary’, output to stderr, and activate all subsequently created WBEM connections for logging:

    configure_loggers_from_string('api=stderr:summary', connection=True)
    
    # All of the following connections will log:
    conn1 = WBEMConnection(...)
    conn2 = WBEMConnection(...)
    
  • Example: Configure both pywbem loggers with the default detail level (‘all’) and output to the file ‘my_log.log’, and activate a single existing WBEM connection object (conn) for logging:

    conn = WBEMConnection(...)
    
    configure_loggers_from_string('all=file', log_filname='my_log.log',
                                  connection=conn)
    

4.9.4. Log records

The following is an example of log output with detail level ‘summary’, where ‘1-32073’ is the connection identifier:

2018-03-17 11:39:09,877-pywbem.api.1-32073-Connection:1-32073 WBEMConnection(url='http://localhost', creds=None, default_namespace='root/cimv2')
2018-03-17 11:41:13,176-pywbem.api.1-32073-Request:1-32073 EnumerateClasses(ClassName=None, DeepInheritance=None, IncludeClassOrigin=None, IncludeQualifiers=None, LocalOnly=None, namespace=None)
2018-03-17 11:41:13,635-pywbem.api.1-32073-Return:1-32073 EnumerateClasses(list of CIMClass; count=103)

The keywords in each log record (‘Connection’, ‘Request’, ‘Return’ or ‘Exception’) identify whether the log record is connection data, API/HTTP request data, API/HTTP response data, or API exception data.

The loggers that actually create the log records are children of the configured pywbem loggers and are unique for each WBEMConnection object. Their logger names are created from the configured logger names by appending the connection identifier (conn_id). Thus the names of the loggers that actually create the log records are of the form: ‘pywbem.<api/http>.<conn_id>’.

Data:

DEFAULT_LOG_DESTINATION

Default log destination if not supplied to the logging configuration functions.

DEFAULT_LOG_DETAIL_LEVEL

Default log detail level if not supplied to the logging configuration functions.

DEFAULT_LOG_FILENAME

Default path name of the log file to be used when logging to a file.

LOGGER_API_CALLS_NAME

Name of the pywbem API logger, which logs user-issued calls to and returns from WBEMConnection methods that drive WBEM operations.

LOGGER_HTTP_NAME

Name of the pywbem HTTP logger, which logs HTTP requests and responses between the pywbem client and WBEM servers.

LOGGER_SIMPLE_NAMES

List of the simple pywbem logger names that the logging configuration functions (see Logging configuration functions) recognize, as follows:

LOG_DESTINATIONS

List of log destinations that the logging configuration functions recognize, as follows:

LOG_DETAIL_LEVELS

List of the log detail levels that the logging configuration functions recognize, as follows:

Functions:

configure_logger(simple_name[, log_dest, ...])

Configure the pywbem loggers and optionally activate WBEM connections for logging and setting a log detail level.

configure_loggers_from_string(...[, ...])

Configure the pywbem loggers and optionally activate WBEM connections for logging and setting a log detail level, from a log configuration string.

pywbem._logging.DEFAULT_LOG_DESTINATION = 'file'

Default log destination if not supplied to the logging configuration functions.

pywbem._logging.DEFAULT_LOG_DETAIL_LEVEL = 'all'

Default log detail level if not supplied to the logging configuration functions.

pywbem._logging.LOGGER_API_CALLS_NAME = 'pywbem.api'

Name of the pywbem API logger, which logs user-issued calls to and returns from WBEMConnection methods that drive WBEM operations.

pywbem._logging.LOGGER_HTTP_NAME = 'pywbem.http'

Name of the pywbem HTTP logger, which logs HTTP requests and responses between the pywbem client and WBEM servers.

pywbem._logging.LOGGER_SIMPLE_NAMES = ['api', 'http', 'all']

List of the simple pywbem logger names that the logging configuration functions (see Logging configuration functions) recognize, as follows:

  • ‘api’ - Pywbem API logger (Python logger name: ‘pywbem.api’)

  • ‘http’ - Pywbem HTTP logger (Python logger name: ‘pywbem.http’)

  • ‘all’ - All pywbem loggers

pywbem._logging.LOG_DESTINATIONS = ['file', 'stderr', 'off']

List of log destinations that the logging configuration functions recognize, as follows:

  • ‘file’ - Log to a file (requires filename to be specified). The file logger appends to the logger file defined by filename.

  • ‘stderr’ - Log to the standard error stream of the Python process.

  • ‘off’ - Disable logging.

pywbem._logging.LOG_DETAIL_LEVELS = ['all', 'paths', 'summary']

List of the log detail levels that the logging configuration functions recognize, as follows:

  • ‘all’ - All of the data available is output. Generally this is the repr() output of the objects in the requests and responses.

  • ‘paths’ - For the API logger, for operations that return CIM classes or CIM instances, only their path component is output as a WBEM URI string. Otherwise, all of the data available is output.

  • ‘summary’ - Only summary information is output. For the API logger this is primarily the count and type of objects returned. For the HTTP logger the HTTP header information is output.

pywbem._logging.configure_logger(simple_name, log_dest=None, detail_level='all', log_filename='pywbem.log', connection=None, propagate=False)[source]

Configure the pywbem loggers and optionally activate WBEM connections for logging and setting a log detail level.

Parameters
  • simple_name (string) –

    Simple name (ex. ‘api’) of the single pywbem logger this method should affect, or ‘all’ to affect all pywbem loggers.

    Must be one of the strings in LOGGER_SIMPLE_NAMES.

  • log_dest (string) –

    Log destination for the affected pywbem loggers, controlling the configuration of its Python logging parameters (log handler, message format, and log level).

    If it is a string, it must be one of the strings in LOG_DESTINATIONS and the Python logging parameters of the loggers will be configured accordingly for their log handler, message format, and with a logging level of DEBUG (see Logging Levels).

    The value ‘off’ disables logging for the affected pywbem loggers.

    If None, the Python logging parameters of the loggers will not be changed.

  • detail_level (string or int or None) –

    Detail level for the data in each log record that is generated by the affected pywbem loggers.

    If it is a string, it must be one of the strings in LOG_DETAIL_LEVELS and the loggers will be configured for the corresponding detail level.

    If it is an int, it defines the maximum size of the log records created and the loggers will be configured to output all available information up to that size.

    If None, the detail level configuration will not be changed.

  • log_filename (string) – Path name of the log file (required if the log destination is ‘file’; otherwise ignored).

  • connection (WBEMConnection or bool or None) –

    WBEM connection(s) that should be affected for activation and for setting the detail level.

    If it is a bool, the information for activating logging and for the detail level of the affected loggers will be stored for use by subsequently created WBEMConnection objects. A value of True will store the information to activate the connections for logging, and will add the detail level for the logger(s). A value of False will reset the stored information for future connections to be deactivated with no detail levels specified.

    If it is a WBEMConnection object, logging will be activated for that WBEM connection only and the specified detail level will be set for the affected pywbem loggers on the connection.

    If None, no WBEM connection will be activated for logging.

  • propagate (bool) – Flag controlling whether the affected pywbem logger should propagate log events to its parent loggers.

Raises

ValueError – Invalid input parameters (loggers remain unchanged).

pywbem._logging.configure_loggers_from_string(log_configuration_str, log_filename='pywbem.log', connection=None, propagate=False)[source]

Configure the pywbem loggers and optionally activate WBEM connections for logging and setting a log detail level, from a log configuration string.

This is most useful in defining the loggers from a command line tool such as pywbemcli so the complete logger configuration can be compressed into a single command line string.

Parameters
  • log_configuration_str (string) –

    The log configuration string, in the following format:

    log_specs := log_spec [ ',' log_spec ]
    log_spec := logger_simple_name [ '=' [ log_dest ] [ ":" [ detail_level ]]]]
    

    where:

  • log_filename (string) – Path name of the log file (required if any log destination is ‘file’; otherwise ignored).

  • connection (WBEMConnection or bool or None) –

    WBEM connection(s) that should be affected for activation and for setting the detail level.

    If it is a bool, the information for activating logging and for the detail level of the affected loggers will be stored for use by subsequently created WBEMConnection objects. A value of True will store the information to activate the connections for logging, and will add the detail level for the logger(s). A value of False will reset the stored information for future connections to be deactivated with no detail levels specified.

    If it is a WBEMConnection object, logging will be activated for that WBEM connection only and the specified detail level will be set for the affected pywbem loggers on the connection.

    If None, no WBEM connection will be activated for logging.

  • propagate (bool) – Flag controlling whether the affected pywbem logger should propagate log events to its parent loggers.

Raises

ValueError – Invalid input parameters (loggers remain unchanged).

Examples for log_configuration_str:

'api=stderr:summary'    # Set 'pywbem.api' logger to stderr output with
                        # summary detail level.

'http=file'             # Set 'pywbem.http' logger to file output with
                        # default detail level.

'api=stderr:summary'    # Set 'pywbem.api' logger to file output with
                        # summary output level.

'all=file:1000'         # Set both pywbem loggers to file output with
                        # a maximum of 1000 characters per log record.

'api=stderr,http=file'  # Set 'pywbem.api' logger to stderr output and
                        # 'pywbem.http' logger to file output, both
                        # with default detail level.

'all=off'               # Disables logging for both pywbem loggers.