e2etest — Framework for writing integration tests for aioxmpp

This subpackage provides utilities for writing end-to-end or intgeration tests for aioxmpp components.

Warning

For now, the API of this subpackage is classified as internal. Please do not test your external components using this API, as it is experimental and subject to change.

Overview

The basic concept is that tests are written like normal unittests. However, tests are written by inheriting classes from aioxmpp.e2etest.TestCase instead of unittest.TestCase. e2etest.TestCase has the provisioner attribute which provides access to a provision.Provisioner instance.

Provisioners are objects which provide a way to obtain a connected XMPP client. The JID to which the client is bound is unspecified; however, each client gets a unique bare JID and the clients are able to communicate with each other. In addition, provisioners provide information about the environment in which the clients act. This includes providing JIDs of entities implementing specific protocols or features. The details are explained in the documentation of the Provisioner base class.

By default, tests which are written with e2etest.TestCase are skipped when using the normal test runners. This is because the provisioners need to be configured; this is handled using a custom nosetests plugin which is not loaded by default (for good reasons). To run the tests, use (instead of the normal nosetests3 binary):

$ python3 -m aioxmpp.e2etest

The command line interface is identical to the one of nosetests3, except that additional options are provided to configure the plugin. In fact, aioxmpp.e2etest is simply a nose test runner with an additional plugin.

By default, the configuration is read from ./.local/e2etest.ini. For details on configuring the provisioners, see the developer guide.

Main API

Decorators for test methods

The following decorators can be used on test methods (including setUp and tearDown):

@aioxmpp.e2etest.require_feature(feature_var, argname=None, *, multiple=False)[source]
Parameters
  • feature_var (str) – XEP-0030 feature var of the required feature

  • argname (str or None) – Optional argument name to pass the FeatureInfo to

  • multiple (bool) – If true, all peers are returned instead of a random one.

Before running the function, it is tested that the feature specified by feature_var is provided in the environment of the current provisioner. If it is not, unittest.SkipTest is raised to skip the test.

If the feature is available, the FeatureInfo instance is passed to the decorated function. If argname is None, the feature info is passed as additional positional argument. otherwise, it is passed as keyword argument using the argname.

If multiple is true, all peers supporting the given feature are passed in a set. Otherwise, only a random peer is returned.

This decorator can be used on test methods, but not on test classes. If you want to skip all tests in a class, apply the decorator to the setUp method.

@aioxmpp.e2etest.require_identity(category, type_, argname=None)[source]
@aioxmpp.e2etest.require_feature_subset(feature_vars, required_subset=[])[source]
@aioxmpp.e2etest.skip_with_quirk(quirk)[source]
Parameters

quirk (Quirks) – The quirk to skip on

If the provisioner indicates that the environment has the given quirk, the test is skipped.

This decorator can be used on test methods, but not on test classes. If you want to skip all tests in a class, apply the decorator to the setUp method.

General decorators

@aioxmpp.e2etest.blocking[source]

The decorated coroutine function is run using the run_until_complete() method of the current (at the time of call) event loop.

The decorated function behaves like a normal function and is not a coroutine function.

This decorator must be applied to a coroutine function (or method).

@aioxmpp.e2etest.blocking_timed[source]

Like blocking_with_timeout(), the decorated coroutine function is executed using asyncio.AbstractEventLoop.run_until_complete() with a timeout, but the timeout is configured in the end-to-end test configuration (see End-to-end tests (or integration tests)).

This is the recommended decorator for any test function or method, to prevent the tests from hanging when anythin goes wrong. The timeout is under control of the provisioner configuration, which means that it can be adapted to different setups (for example, running against an XMPP server in the internet will be slower than if it runs on localhost).

The decorated function behaves like a normal function and is not a coroutine function.

This decorator must be applied to a coroutine function (or method).

@aioxmpp.e2etest.blocking_with_timeout(timeout)[source]

The decorated coroutine function is run using the run_until_complete() method of the current (at the time of call) event loop.

If the execution takes longer than timeout seconds, asyncio.TimeoutError is raised.

The decorated function behaves like a normal function and is not a coroutine function.

This decorator must be applied to a coroutine function (or method).

Class for test cases

class aioxmpp.e2etest.TestCase(methodName='runTest')[source]

A subclass of unittest.TestCase for end-to-end test cases.

This subclass provides a single additional attribute:

provisioner

This is the configured provision.Provisioner instance.

If no provisioner is configured (for example because the e2etest nose plugin is not loaded), this reads as None.

Note

Under nosetests and the vanilla unittest runner, tests inheriting from TestCase are automatically skipped if provisioner is None.

Provisioners

class aioxmpp.e2etest.provision.Provisioner(logger=<Logger aioxmpp.e2etest.provision (WARNING)>)[source]

Base class for provisioners.

Provisioners are responsible for providing test cases with XMPP accounts and client objects connected to these accounts, as well as information about the environment the accounts live in.

A provisioner must implement the following methods:

abstract configure(section)[source]

Read the configuration and set up the provisioner.

Parameters

section – mapping of config keys to values

Subclasses will implement this to configure their account setup and servers to use.

See also

configure_tls_config()

for a function which extracts TLS-related arguments for aioxmpp.security_layer.make()

configure_quirks()

for a function which extracts a set of Quirk enumeration members from the configuration

configure_blockmap()

for a function which extracts a mapping which allows to block features from specific hosts

The following methods are the API used by test cases:

get_feature_provider(feature_nses)[source]
Parameters

feature_ns (iterable of str) – Namespace URIs to find a provider for

Returns

JID of the entity providing all features

Return type

aioxmpp.JID

If there is no entity supporting all requested features, None is returned.

get_identity_provider(category, type_)[source]
has_quirk(quirk)[source]
Parameters

quirk (Quirk) – Quirk to check for

Returns

true if the environment has the given quirk

These methods can be used by provisioners to perform plumbing tasks, such as shutting down clients or deleting accounts:

class aioxmpp.e2etest.provision.AnonymousProvisioner[source]

This provisioner uses SASL ANONYMOUS to obtain accounts.

It is dead-simple to configure: it needs a host to connect to, and optionally some TLS and quirks configuration. The host is specified as configuration key host, TLS can be configured as documented in configure_tls_config() and quirks are set as described in configure_quirks(). A configuration for a locally running Prosody instance might look like this:

[aioxmpp.e2etest.provision.AnonymousProvisioner]
host=localhost
no_verify=true
quirks=[]

The server configured in host must support SASL ANONYMOUS and must allow communication between the clients connected that way. It may provide PubSub and/or MUC services, which will be auto-discovered if they are provided in the XEP-0030 items of the server.

class aioxmpp.e2etest.provision.AnyProvisioner[source]

This provisioner randomly generates usernames and uses a hardcoded password to authenticate with the XMPP server.

This is for use with mod_auth_any of prosody.

It is dead-simple to configure: it needs a host to connect to, and optionally some TLS and quirks configuration. The host is specified as configuration key host, TLS can be configured as documented in configure_tls_config() and quirks are set as described in configure_quirks(). A configuration for a locally running Prosody instance might look like this:

[aioxmpp.e2etest.provision.AnyProvisioner]
host=localhost
no_verify=true
quirks=[]

The server configured in host must allow authentication with any username/password pair and allow communication between the clients connected that way. It may provide PubSub and/or MUC services, which will be auto-discovered if they are provided in the XEP-0030 items of the server.

class aioxmpp.e2etest.provision.StaticPasswordProvisioner[source]

This provisioner expects a list of username/password pairs to authenticate against the tested server.

This is for use with servers which support neither SASL ANONYMOUS nor a mod_auth_any equivalent.

The configuration of this provisioner is slightly unwieldly since we do not want to add a dependency to a more sane configuration file format. Here is an example on how to configure a provisioner with two accounts:

[aioxmpp.e2etest.provision.StaticPasswordProvisioner]
host=localhost
accounts=[("user1", "password1"), ("user2", "password2")]
skip_on_too_few_accounts=false

All accounts need to have exactly the same privileges on the server. The first account will be used to auto-discover any features offered by the test environment.

If skip_on_too_few_accounts is set to true (the default is false), tests will be skipped if the provisioner runs out of accounts instead of failing.

class aioxmpp.e2etest.Quirk(value)[source]

Enumeration of implementation quirks.

Each enumeration member represents a quirk of an implementation. A quirk is a behaviour of an implementation which does not directly violate standards, but which is unfortunate in a way that it disables some features of aioxmpp.

One example of such a quirk is the rewriting of message stanza IDs which some MUC implementations do when reflecting the messages. This breaks the stanza tracking of aioxmpp.muc.Room.send_tracked_message().

The following quirks are defined:

MUC_REWRITES_MESSAGE_ID https://zombofant.net/xmlns/aioxmpp/e2etest/quirks#muc-id-rewrite

This quirk must be configured when the environment the provisioner provides rewrites the message IDs when they are reflected by the MUC implementation.

The quirk does not need to be set if the environment does not provide a MUC implementation at all.

PUBSUB_GET_ITEMS_BY_ID_BROKEN https://zombofant.net/xmlns/aioxmpp/e2etest/quirks#broken-pubsub-get-multiple-by-id

Indicates that the “Get Items by Id” operation in the PubSub service used is broken when more than one item is requested.

Helper functions

aioxmpp.e2etest.provision.configure_tls_config(section)[source]

Generate keyword arguments for use with security_layer.make() from the configuration which control the TLS behaviour of the security layer.

Parameters

section – Configuration section to work on.

Returns

Keyword arguments for security_layer.make()

Return type

dict

The generated keyword arguments are pin_type, pin_store and no_verify. The options in the config file have the same names and the semantics are the following:

pin_store and pin_type can be used to configure certificate pinning, in case the server you want to test against does not have a certificate which passes the default OpenSSL PKIX tests.

If set, pin_store must point to a JSON file, which consists of a single object mapping host names to arrays of strings containing the base64 representation of what is being pinned. This is determined by pin_type, which can be 0 for Public Key pinning and 1 for Certificate pinning.

There is also the no_verify option, which, if set to true, will disable certificate verification altogether. This does not much harm if you are testing against localhost anyways and saves the configuration nuisance for certificate pinning. no_verfiy takes precedence over pin_store and pin_type.

aioxmpp.e2etest.provision.configure_quirks(section)[source]

Generate a set of Quirk enum members from the given configuration section.

Parameters

section – Configuration section to work on.

Returns

Set of Quirk members

This parses the configuration key quirks as a python literal (see ast.literal_eval()). It expects a list of strings as a result.

The strings are interpreted as Quirk enum values. If a string starts with #, it is prefixed with https://zombofant.net/xmlns/aioxmpp/e2etest/quirks for easier manual writing of the configuration. See Quirk for the currently defined quirks.