Update: gabbi-tempest is now documented and fully merged up to make the ideas below work. I wrote an email to the openstack-dev explaining the current state of affairs. I also found a post from January describing an earlier stage in the process.

Imagine being able to add integration API tests to an OpenStack project by creating a directory and adding a YAML file in that directory that is those tests. That's the end game of what I'm trying to do with gabbi-tempest and some experiments with zuul jobs.

Gabbi is a testing tool for HTTP APIs that models the requests and responses of a series of HTTP requests in a YAML file. Gabbi-tempest integrates gabbi with tempest, an integration test suite for OpenStack, to provide some basic handling for access to the service catalog and straightforward authentication handling within the gabbi files. Tempest sets up the live services, and provides some images and flavors to get you started.

Here's a simple example that sets some defaults for all requests and then verifies the version discovery doc for the placement service using JSONPath:

defaults:
    request_headers:
        x-auth-token: $ENVIRON['SERVICE_TOKEN']
        content-type: application/json
        accept: application/json
        openstack-api-version: 'compute latest, placement latest'
        verbose: True

tests:

    - name: get placement version
      GET: $ENVIRON['PLACEMENT_SERVICE']
      response_json_paths:
          $.versions[0].id: v1.0

This is used in a work-in-progress change to placement. It is a work in progress because there are several pieces which need to come together to make the process as clean and delightful as possible.

The desired endpoint is that for a project to turn this kind of testing on they would:

1. Add an entry to templates in their local .zuul.yaml, something like openstack-tempest-gabbi. This would cause a few different things:

   • openstack-tempest-gabbi jobs added to both gate and check.
   • That job would set or extend the GABBI_TEMPEST_PATH environment variable to include a gabbits directory from the code checkout of the current service. That environment variable defines where the gabbi-tempest plugin looks for YAML files.
   • And then run tempest:
     • with the gabbi-tempest plugin
     • With tempest_test_regex: 'gabbi' to limit the tests to just gabbit tests (not necessary if other tempest tests are desired).
     • With tox_envlist: all, which is the tox environment that is the current correct choice when wanting to use a test regex.

2. Create a directory in their repo, perhaps gabbits and put one or more gabbi YAML files like the example above in there.

We're some distance from that but there are pieces in progress that move things in that direction. I'm hoping that the above gives a better description of what I'm hoping to achieve and encourages people to help, because I need some.

Some of the in progress pieces:

• WIP: Create a gabbi-tempest zuul job
• WIP: Use gabbi-tempest job from tempest
• gabbi-tempest, which needs to become an openstack thing