Update: gabbi-tempest is now documented and fully merged up to make the ideas below work. I wrote an email to the openstack-dev explaining the current state of affairs. I also found a post from January describing an earlier stage in the process.
Imagine being able to add integration API tests to an OpenStack project by creating a directory and adding a YAML file in that directory that is those tests. That's the end game of what I'm trying to do with gabbi-tempest and some experiments with zuul jobs.
Gabbi is a testing tool for HTTP APIs that models the requests and responses of a series of HTTP requests in a YAML file. Gabbi-tempest integrates gabbi with tempest, an integration test suite for OpenStack, to provide some basic handling for access to the service catalog and straightforward authentication handling within the gabbi files. Tempest sets up the live services, and provides some images and flavors to get you started.
Here's a simple example that sets some defaults for all requests and then verifies the version discovery doc for the placement service using JSONPath:
defaults: request_headers: x-auth-token: $ENVIRON['SERVICE_TOKEN'] content-type: application/json accept: application/json openstack-api-version: 'compute latest, placement latest' verbose: True tests: - name: get placement version GET: $ENVIRON['PLACEMENT_SERVICE'] response_json_paths: $.versions.id: v1.0
This is used in a work-in-progress change to placement. It is a work in progress because there are several pieces which need to come together to make the process as clean and delightful as possible.
The desired endpoint is that for a project to turn this kind of testing on they would:
Add an entry to
templatesin their local
.zuul.yaml, something like
openstack-tempest-gabbi. This would cause a few different things:
openstack-tempest-gabbijobs added to both gate and check.
- That job would set or extend the
GABBI_TEMPEST_PATHenvironment variable to include a
gabbitsdirectory from the code checkout of the current service. That environment variable defines where the
gabbi-tempestplugin looks for YAML files.
- And then run tempest:
- with the
tempest_test_regex: 'gabbi'to limit the tests to just gabbit tests (not necessary if other tempest tests are desired).
tox_envlist: all, which is the tox environment that is the current correct choice when wanting to use a test regex.
- with the
Create a directory in their repo, perhaps
gabbitsand put one or more gabbi YAML files like the example above in there.
We're some distance from that but there are pieces in progress that move things in that direction. I'm hoping that the above gives a better description of what I'm hoping to achieve and encourages people to help, because I need some.
Some of the in progress pieces: