2. Concepts

This sections defines some of the basic concepts that form the basis for Pywbem including the architecture, basic CIM/WBEM components, operations and indications.

It also defines the differences between some of the methods defined in Pywbem cim_operations, in particular, the pull operations and the iter operations.

2.1. The CIM/WBEM architecture

ISSUE #2669: Write this section on CIM/WBEM architecture

2.2. The CIM model and CIM objects

ISSUE #2669: Write this section on the model and CIM objects

2.3. WBEM Operations: Communicating with the WBEM Server

ISSUE #2669: Write this section on WBEM Operations

2.3.1. WBEM operations overview

ISSUE #2669 Write this section on WBEM operations overview

2.3.2. Traditional operations

ISSUE #2669 - Write this section on traditional operations

2.3.3. Pull operations

The DMTF CIM/XML pull operations allow the WBEM client to break the monolithic instance operations for requests that deliver multiple objects into multiple requests/responses executed as a sequence of requests to limit the size of individual responses.

NOTE: The pull operations were added to pywbem in version 0.9.0.

They were created to reduce scalability issues with extremely large responses from servers for instance enumerate operations (EnumerateInstances, Associators, and References) that were causing resource problems in both clients and servers. The pull operations allow the client to break large responses up into multiple smaller responses and allowing filtering of the responses to be performed by the server.

A central concept of pulled enumeration operations is the enumeration session, which provides a context in which the operations perform their work and which determines the set of instances or instance paths to be returned. To process the operations of an enumeration session, some parameters of the open operation need to be maintained as long as the enumeration session is open. In addition, some state data about where the enumeration session is with regard to instances or instance paths already returned must be maintained.

A successful Open… operation establishes the enumeration session and returns an enumeration context (Pybbem result.context) value representing that session. This value is used as a parameter in subsequent Pull operations on that enumeration session.

In general the pull operations replace the single monolithic request/response (ex. EnumerateInstances) that returns all instances in a single response with a pattern (enumeration sequence) that is based on three operations:

  • Open… to open the request enumeration session to the WBEM Server and optionally request objects be returned.

  • Pull… to continue retrieving objects from the WBEM Server after a successful Open…. The client normally continues to execute pulls until an exception or end-of-sequence flag is received.

  • CloseEnumeration Close an enumeration sequence before it is complete. This request is ONLY used to execute early close; when the eos flag is returned or and error is returned (if ContinueOnError is not set), the session is closed by the server.

The open… requests use the same request parameters as the traditional operations to define the basic characteristics of the corresponding traditional operation (object name, propertylists, etc) and add several more request parameters to control the flow of responses (response size, timeouts, etc.). In addition they add two new request parameters (QueryFilter and QueryFilterLanguage) to request that the server filter the responses and return only instances/paths that match the filter.

2.3.3.1. Relation to traditional operations

The convention for the pull operation naming is as follows:

  1. Prepend the traditional operation name with Open and Pull

  2. Suffix the pull operations that return both instances and paths with WithPath

  3. Change the name suffix on operations that return path information from Names to Paths to reflect that these operations are returning complete instance paths with host and namespace included. The Exec was dropped from the name for the OpenQueryInstances.

The pull operations parallel to the traditional operations as follows:

Traditional Operation

Pull Operations

EnumerateInstances

OpenEnumerateInstances / PullInstancesWithPath

EnumerateInstanceNames

OpenEnumerateInstancePaths / PullInstancePaths

Associators

OpenAssociatorInstances / PullInstancesWithPath

AssociatorNames

OpenAssociatorInstancePaths / PullInstancePaths

References

OpenReferenceInstances / PullInstancesWithPath

ReferenceNames

OpenReferenceInstancePaths / PullInstancePaths

ExecQuery

OpenQueryInstances / PullInstances

The pull operations are defined only for instances. There are NO pull operations for CIM classes, the for CIM qualifier declarations or for method invocations.

2.3.3.2. Pull operation responses

Each pull operation request returns a Python namedtuple result that consists of the following named components:

  • eos - A boolean flag that indicates the end of the enumeration sequence. As along as this returned flag is false, the server has more objects to return. If this flag is true in a response, the server has no more objects to return and it has closed the enumeration sequence.

  • context - An opaque identifier that must be returned to the server with subsequent pull requests to continue the enumeration sequence. The context received with a response within an enumeration must be returned with the next request since the context may uniquely define not only the enumeration sequence but the segement returned in the response.

  • instances or paths - A list of pywbem objects returned from the server. The requests that demand instances return the instances entry in the namedtuple and those that request paths return paths in the path entry in the namedtuple.

2.3.3.3. Pull enumeration sequence code pattern

Generally the pattern for requesting from a server using the pull operations is as follows:

# open the enumeration sequence
result = open...(uri, ...)
    ... process the objects return in result.xx
# while more objects exist in the server, loop to pull objects
while not result.eos
    result = pull...(result.context, <MaxObjectCount>, ...)
        ... process the objects return in result.xx

The user opens the request with the open request and if that is successful, and does not return the end-of-sequence flag the result (eos) executed the pull request to continue receiving objects within the enumeration sequence. Each pull request MUST include the enumeration context from the previous response (context in the result tuple).

The pull sequence may be terminated by executing a [CloseEnumeration()](https://pywbem.readthedocs.io/en/latest/client.html#pyw bem.WBEMConnection.CloseEnumerate) to terminate the pull sequence. However, this is optional and used only to close pull sequences before the eos has been received.

2.3.3.4. Common Pull Operation Request Input Arguments

The following are the request arguments that are common across all of the Pull requests.

2.3.3.4.1. Open requests
  • FilterQuery Language and FilterQuery - These input parameters specify a filter query that acts as an additional restricting filter on the set of enumerated instances/paths returned. WBEM servers must support filter queries in pulled enumerations and must support the DMTF Filter Query Language(FQL, see DMTF DSP0212) as a query language. If a WBEM server accepts a request with the FilterQuery parameter defined it MUST filter the response. NOTE: The query and query language defined for the OpenQueryInstances is NOT FQL but the same query languages defined for the execQuery request.

  • OperationTimeout - Determines the minimum time the WBEM server shall maintain the opened enumeration session after the last Open or Pull operation (unless the enumeration session is closed during the last operation). If the operation timeout is exceeded, the WBEM server may close the enumeration session at any time, releasing any resources allocated to the enumeration session. An OperationTimeout of 0 means that there is no operation timeout. That is, the enumeration session is never closed based on time. If OperationTimeout is NULL, the WBEM server shall choose an operation timeout.

  • ContinueOnError - This input parameter, if true, requests a continuation on error, which is the ability to resume an enumeration session successfully after a Pull operation returns an error. If a WBEM server does not support continuation on error and ContinueOnError is true, it shall return a failure with the status code CIM_ERR_CONTINUATION_ON_ERROR_NOT_SUPPORTED. Most servers today do not support ContinueOnError.

  • MaxObjectCount - Defines the maximum number of instances or instance paths that the open operation can return. Any uint32 number is valid, including 0. The WBEM server may deliver any number of instances or instance paths up to MaxObjectCount but shall not deliver more than MaxObjectCount elements. The default for this is zero so that the WBEM server does not deliver objects in the response unless a MaxObjectCount is specifically defined. The WBEM server may limit the maximum size of this request parameter.

2.3.3.4.2. Pull requests
  • Context - This is the EnumerationContext defined in the specification. It is an opaque string returned from the previous open or pull for this enumeration sequence as part of the result tuple (result.context).

  • MaxObjectCount - This required input parameter defines the maximum number of instances or instance paths that may be returned by this Pull operation. Any uint32 number is valid, including 0. The WBEM server may deliver any number of instances or instance paths up to MaxObjectCount but shall not deliver more than MaxObjectCount. The WBEM client may use a MaxObjectCount value of 0 to restart the operation timeout for the enumeration session when it does not need to not retrieve any instances or instance paths.

2.3.3.4.3. Close request
  • Context - This is the EnumerationContext defined in the specification. It is an opaque string returned from the previous open or pull for this enumeration sequence as part of the result tuple (result.context).

2.3.3.5. Differences from traditional operations

The pull operations differ from the traditional operations in the several ways:

  1. They allow filtering the response in the WBEM Server which can represent a significant resource saving if only selected instances from a large response are really required.

  2. They limit the amount of memory used by the server since the server need not process the complete request before returning information to the client

  3. They limit the memory used by the client since it can define the maximum size of any response.

  4. They allow the client to terminate an enumeration early with the CloseEnumeration.

  5. They allow the server and client to receive partial responses in that the client receives potentially an error response on each segment of the response, not the overall response.

  6. They provide a more consistent inclusion of the path component in the responses.

2.3.4. Iter operations

The iterable operation extensions (short: iter operations) are a set of methods added to [pywbem.WBEMConnection](https://pywbem.readthedocs.io/en/latest/client.html# pywbem.WBEMConnection) class in pywbem version 0.10.0 to simplify the use of the pull vs. traditional operations.

These are specific to PyWBEM.

2.3.4.1. Why the iter operations exist

The iter operations provide:

1. An interface that is the same whether the user is executing the pull operations or their equivalent traditional operations.

2. An interface that use the Python iterator paradigm to get instances or instance paths in place of lists or tuples as for the pull operations and traditional operations.

3. An interface that allows the user to utilize pull operations or traditional operations with just an attribute change in WBEMConnection.

4. An interface that automatically attempts to use pull operations and if a particular WBEM server does not support them falls back to the equivalent traditional operations so the user does not need to worry about whether the server supports the pull operations or if they are required for memory optimization.

2.3.4.2. Comparison table

The traditional operations and their equivalent pull operations are covered by the new iter operations as follows:

Traditional Operation

Pull Operations

Iter Operation

EnumerateInstances

OpenEnumerateInstances / PullInstancesWithPath

IterEnumerateInstances

EnumerateInstanceNames

OpenEnumerateInstancePaths / PullInstancePaths

IterEnumerateInstancePaths

Associators

OpenAssociatorInstances / PullInstancesWithPath

IterAssociatorInstances

AssociatorNames

OpenAssociatorInstancePaths / PullInstancePaths

IterAssociatorInstancePaths

References

OpenReferenceInstances / PullInstancesWithPath

IterReferenceInstances

ReferenceNames

OpenReferenceInstancePaths / PullInstancePaths

IterReferenceInstancePaths

ExecQuery

OpenQueryInstances / PullInstances

IterQueryInstances

The methods for the iter operations use the same arguments as the Open… methods of the pull operations, with exceptions noted in section :ref: Differences between iter operations and pull operations.

The general pattern for use of the iter operations is:

try:
    iterator = Iter...(...)
    for object in iterator:
        <process the object>
except Error as er:
    # NOTE: objects may be received before an exception, because in each call
    # the server returns either objects or error. However, generally the
    # first error terminates the whole sequence.

These operations use the Python iterator paradigm so that the for-loop processes CIM objects as they are received via the pull operations or via the traditional operations if the server does not support pull operations.

2.3.4.3. Internal processing in the iter operations

The iter operations try to use the existing pull operations or traditional operations and lay a layer over them to determine if the pull operations can be used and to manage the iteration. The paradigm for the implementation of each of these operations is generally as follows (showing an operation returning instances as an example, and omitting the logic that closes the pull operation):

# psuedo code pattern for iter function internal processing
if <use_pull_for_this_operation is try or true>:
    try:
        result = Open...(...)
        <use_pull_for_this_operation = true>
        for inst in result.instances:
            yield inst
        while not result.eos:
            result = PullInstancesWithPath(...)
            for inst in result.instances:
                yield inst
        return
    except CIMError as ce:
        if <use_pull_for_this_operation is try> and
                ce.status_code != "CIM_ERR_NOT_SUPPORTED":
            <use_pull_for_this_operation = false>
        else:
            raise
<check for unsupported parameters when using traditional operations>
instances = <traditional-operation>(...)
for inst in instances:
    <fix up path in instance>
    yield inst

2.3.5. Forcing pull vs. traditional operations

A parameter (use_pull_operations) has been added to the [pywbem.WBEMConnection](https://pywbem.readthedocs.io/en/latest/client.html# pywbem.WBEMConnection) constructor to optionally force the use of either the pull operations or the traditional operations.

  • If use_pull_operations is True only the pull operations will be executed and if this fails for any reason including CIM_ERR_NOT_SUPPORTED, the exception will be returned.

  • If use_pull_operations is False only the traditional operations will be executed and if this fails for any reason, the exception will be returned.

  • The default is None. In this case, first the pull operation will be attempted. If the first request (Open…) returns CIM_ERR_NOT_SUPPORTED, the corresponding traditional operation will be attempted.

Thus, the iter operations can be used to execute exclusively the traditional operations by simply setting use_pull_operations=False.

conn = pywbem.WBEMConnection(server, (username, password),
                             default_namespace=namespace,
                             no_verification=True,
                             use_pull_operations=False)

.._Differences between iter operations and pull operations:

2.3.6. Differences between iter operations and pull operations

2.3.6.1. Use of FilterQuery

Since the traditional operations did not incorporate the query filters into their input parameters, if a query filter is included in the request and the request is passed to a traditional operation, the request will be refused and an exception generated. This is because the specification for the FilterQuery states that the server must return filtered responses and there is no way to do that with the traditional operations.

2.3.6.2. Paths in returned instances

The requirements on paths in returned instances differ between pull and traditional operations. The iter operations have been defined to be in line with the requirements on paths for pull operations, and the implementation of the iter operations acts to bring the path in returned instances in line with the requirements of the pull operations, if it uses the traditional operation. Thus, the iter operation always returns a complete path in any returned instances.

2.3.6.3. Use of MaxObjectCount argument

The MaxObjectCount argument is somewhat more limited than if the pull operations are used directly in that:

  1. It is the same value for open and pull requests.

  2. The mechanism to delay responses (setting MaxObjectCount=0 and executing a Pull…() method) cannot be used so the interoperation timeout must be sufficient for the client to complete its processing.

2.3.6.4. Receiving returned objects before an exception

In general the pull operations receive either objects or error for each request (open or pull). Since these operations may be called to get objects from the server the iterator may receive objects before an exception is executed. In general, unless the ContinueOnError flag is set, the enumeration sequence will terminate after the first error and that error is an indication that not all objects were received from the server. If the traditional enumerate function is called by the Iter…() method, either objects or an error are received, never both.

2.3.6.5. Closing an Iter operation before it is complete

An iter operation may be closed before the processing from the server is complete by executing the close() function on the iterator:

inst_iterator = conn.IterEnumerateInstances(classname,
                                            MaxObjectCount=max_obj_cnt)
for inst in inst_iterator:
    if <instance fails some test>
        inst_iterator.close()
    else:
        <process the instance>

Note that if the operation executed was the traditional operation rather than the pull operation, the close() will do nothing since the response instances are received as a single block. If the enumeration sequence is already complete, this call will also be ignored.

2.4. WBEM indications and subscriptions

ISSUE #2669: write this Section on indications and subscriptions

2.5. WBEM Management Profiles

ISSUE #2669: Create this section describing profiles, why there exist and very generally how to use them