There has been a lot of thinking recently about the next steps for gabbi. The ideas being considered are mostly associated with making gabbi more flexible and more extensible and will introduce some backwards incompatible features. Thus there will need to be a version 2.0.
When gabbi was born it was created to solve a particular problem with testing the JSON API in OpenStack ceilometer. That context introduced several specific constraints stemming from one major constraint: The tests created by gabbi had to work in the "standard" OpenStack environment. That meant the tests:
- had to be relatives of
- had to be discoverable and individually selectable by testrepository
- were completely oriented towards sending and receiving
All the initial development work on gabbi set out to satisfy these requirements. This has led to some limitations that should be resolved:
Gabbi tests require a test runner that supports the load tests protocol. This precludes working with
py.testand initially made it hard to make gabbi-run.
Slightly more indirection and abstraction in the test generation process may make it possible to generate tests for both
load_testsas well as make the interface between
gabbi-runand the rest of gabbi more tidy.
Processing of data, in the body of both requests and responses, is oriented toward JSON which makes it hard to use gabbi to test services that use
HTML, web forms,
XMLor other technologies. While there is support for extensions called response handlers these are too limited: They only add ways to validate a response body and do not provide tooling for processing YAML into properly formed request bodies nor referring to prior responses in subsequent requests (a subsystem referred to in the code as
replacers). Tooling for these features is built in for JSON.
The proposal for solving this problem is a concept called ContentHandlers. These will encapsulate three pieces of functionality in external packages that hopefully will be easy for other people to create and distribute for use as extensions to gabbi. The functionality is:
Evaluating response bodies to verify the content. This is the same functionality provided by
response_json_pathsin the current gabbi structure and extensible today by response handlers.
Massaging the value of the
datakey in a gabbi test to transform it into a variety of respresentations. Right now the value is one of a string, a byte string, or a JSON object. It would be useful to also cogently represent other structures such as url encoded web form data.
Adding functionality to the
replacersthat drive the substitutions in
$RESPONSEhandling. This currently works with only JSONPath but there are many other response bodies which are queryable.
Each of these are driven by the
content-typeof the request or the response so that value will drive how
ContentHandlerextensions are accessed by the core code.
FND has been doing some exploratory work with gabbi-html to figure out the issues gabbi will need to overcome to have easy to use extensions for different content types.
There is as yet no timetable on this work, nor is it set in stone. If
you have suggestions, questions, or want to help, leave a comment here,
make an issue, or join us in
#gabbi on freenode IRC.