Index ¦ Archives ¦ Atom

OpenStack Developer Satisfaction

When I ran for the OpenStack Technical Committee one of the roles I took for myself was something like a union rep for the people who are existing developers. The people who are there multiple times per week and self-identify as OpenStack developers. Earlier this year I started noticing (more than normal) that sometimes things are less than ideal for these people. This blog post tries to gather some thoughts on this topic as well as present some analysis of survey data. The survey analysis was presented at the recent leadership meeting at the OpenStack Summit in Sydney.

But first a caveat or disclaimer to get out of the way: Being a paid open source developer is a pretty great role that is typically only available to privileged people and achieving the role frequently gives those people yet more privilege. The job is many times better than many other options so when considering these issues of "satisfaction" it is important to keep in mind that this is relative to an ideal. The frustration that some people express is not because everything sucks but because the difference between what is happening and what could be happening is fraught with frustration. At the same time, the pressures which people experience—either because they impose it themselves or feel it from something external—is real and the mental and physical health implications are real. Members of any community, even one as privileged as OpenStack, should work together to limit dissatisfaction and pressures. If successful, the community may grow in a way that makes it more inclusive.

With that out of the way, the observations I had can be grouped in a few different ways, but mostly come down to either being too included (in the vortex of vast amounts of work and commitment) or not being included enough.

Some developers, often those who are unicorn "100% upstream" style developers, are obliged to act heroically to bring work to completion. They maintain continuity and feel such a commitment to the project that they fill all voids. Some even enjoin newbies to "double down" on their efforts, while others suggest that success upstream comes from catching up after normal business hours.

Other developers, either new ones or ones that do not wish to overcommit, are effectively excluded: while the heroes turn inward to each other, everyone else is on the fringes, unable to engage because of impedance mismatch.

In either case there is exhaustion: In the last year, I have heard from multiple contributors who, when presented with the opportunity to leave OpenStack, have leapt at the chance saying "thank god!". They are exhausted by trying to negotiate the complex social processes, the labyrinthine code, the glacial pace of review and difficulties gaining traction (when you are not "in"). It's a process which is not fun. It is riddled with exclusionary complexity not just for new people, but for people who are compelled to be around for whatever reason.

At the Board

With that as background, at the PTG in Denver I approached the Foundation board to say that I didn't think that "developer happiness" was sufficiently on the strategic radar. There were efforts in progress to welcome new contributors and to groom new leaders but little for the daily laborer. A couple things happened as a result:

  • I learned that this topic was within the domain of "community health" (one of 5 overarching strategic themes) but had not had much attention yet due to lack of people. Thus I was volunteered.

  • Two questions were added to the survey given to PTG attendees to gauge satisfaction. See the results.

Those two questions were:

  • What is the most important thing we should do to improve the OpenStack software over the next year?

  • What is one change that would make you happier as an OpenStack Upstream developer?

I had asked for a question that was yes or no, or at least a Likert scale (something like "If presented with a reasonable opportunity to leave the OpenStack community would you take it? 1-7, disagree-agree" to inspect the "thank god!" issue described above). Something where the responses could be analysed in a relatively unbiased way. Since the answers to the chosen questions are completely open-ended the best option for analysis is thematic analysis. I did that in this spreadsheet, applying tags to each response. The presentation has some summary analysis.

Prior to reviewing the survey results I had made a list of the issues I thought were present in the topic of "developer happiness":

  • too much clubbiness in some groups
  • code quality makes people not want to participate
  • project velocity make people not want to participate
  • corporate commitment limited
  • too much work, overall
  • legacy code and architecture that is weird

The first four were well represented in the survey results. The three main themes identified as areas of concern were:

  • Developer population, subdivided into:
    • Corporate involvement: concerns about shrinking commitments from traditional corporate participants
    • Developer quantity: bald statements of "we need more"
    • Developer retention: treatment, recognition, people leaving
    • Technical leadership: respondents asking for increased, or at least unified, voices on direction project-wide
    • Labor issues: not enough time to get things done, too much distraction from other responsibilities, conflicts in upstream community
  • Product Focus: What the projects should be working on
    • Refinement: making what's there the best it can be
    • Simplify: limiting the number of knobs
    • Stability: making what's there actually work
    • Upgradability: get upgrades working
    • Pro-Features: add new stuff
    • Pro-Fixes: Fix what's there, pay down debt
  • Clubs: People expressing concern with the difficulty of getting attention or breaking into perceived cliques.

The product focus theme was pretty stark. There were two responses that could be interpreted as "pro-feature" versus 56 that could be interpreted as "pro fix what's there". In general many of the responses could be interpreted as "this could be good if we cleaned up after ourselves". It is important, however, to take that attitude with a grain of salt. I think every large software project ever has presented that face at some point.

Product Focus

The clubbiness is a huge concern. We frequently speak about wanting OpenStack to be inclusive across many cultures and to many types of contributions. Many people perceive that there are strong groups in OpenStack and some of them are difficult to break into. Some of this is the result of technical learning curves, but some of it is social.

Overall it was challenging to get a specific read on what needs to change to make the experience of OpenStack developers happier or more satisfied. It was clear from the analysis that a lot of people want things to change but often that change is in conflicting directions. In the image below "Negative" does not mean "everything sucks" but rather that something is not how a respondent would like it to be.

General Feel

Based on my analysis of the survey results I made a few recommendations to the board for where we should focus additional attention:

  • Continue increasing focus on cross-project (activity and recognition)
  • Enhance focus on fixing what exists and paying down tech debt
  • More technical leadership
  • Faster creation of more cores
  • Do something about Nova (identified as a leading culprit where people experience difficulty "breaking in", something that has been talked about for years)
  • Ensure higher fidelity communication between upstream devs, and management and resourcing decision makers (there's a mistaken belief that upstream contributors are effectively communicating about their upstream experience to their internal management)
  • Keep having piano bar at PTG

Some of these are already in progress, others will get started in 2018. Suggestions or counterpoints are of course welcome. For now this is still in the information gathering and analysis stage, it would be premature to make any dramatic decisions based on the info gathered thus far.

I hope that by broaching the topic we can make it a bit less taboo and start working to find solutions at both the community and individual level to allow contributors to be productive and happy in a sustainable fashion. Even if the number of heroes burning themselves out and the number of contributors yearning for change is small, any number is too big.

© Chris Dent. Built using Pelican. Theme by Giulio Fidente on github.