Write//anticdent.org/2022-12-20T11:41:00+00:00Checking In2022-12-20T11:41:00+00:002022-12-20T11:41:00+00:00Chris Denttag:anticdent.org,2022-12-20:/checking-in.html<p>This is a brief check in to indicate that I (and this blog) am still alive.</p>
<p>There's a joke in my family that I'm responsible for inventing the internet.
This is obviously not true, but my involvement with various pre-Web protocols,
the emergence of the Web, and transitioning a town's …</p><p>This is a brief check in to indicate that I (and this blog) am still alive.</p>
<p>There's a joke in my family that I'm responsible for inventing the internet.
This is obviously not true, but my involvement with various pre-Web protocols,
the emergence of the Web, and transitioning a town's ISP into the Web era
makes it a convenient foil against which I am blamed (or take responsibility)
for modern ills of attention and vacuity.</p>
<p>But it's capitalism, not me. The engine which gives me a fat salary, a nice
place to live and a deep sense of guilt and injustice. That's what's to blame
for so much that we see as illness today.</p>Fix Your Debt: Placement Performance Summary2019-10-04T14:32:00+01:002019-10-04T14:32:00+01:00Chris Denttag:anticdent.org,2019-10-04:/fix-your-debt-placement-performance-summary.html<p>There's a thread on the openstack-discuss mailing list, started in
<a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009832.html">September</a>
and then continuing in
<a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-October/thread.html#9835">October</a>,
about limiting planned scope for Nova in the Ussuri cycle so that
stakeholders' expectations are properly managed. Although Nova gets
a vast amount done per cycle there is always some stuff left undone
and …</p><p>There's a thread on the openstack-discuss mailing list, started in
<a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009832.html">September</a>
and then continuing in
<a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-October/thread.html#9835">October</a>,
about limiting planned scope for Nova in the Ussuri cycle so that
stakeholders' expectations are properly managed. Although Nova gets
a vast amount done per cycle there is always some stuff left undone
and some people surprised by that. In the midst of the thread,
Kashyap <a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-October/009924.html">points
out</a>:</p>
<blockquote>
<p>I welcome scope reduction, focusing on fewer features, stability,
and bug fixes than "more gadgetries and gongs". Which also means: less
frenzy, less split attention, fewer mistakes, more retained
concentration, and more serenity. [...] If we end up with bags of
"spare time", there's loads of tech-debt items, performance (it's a
feature, let's recall) issues, and meaningful clean-ups waiting to
be tackled.</p>
</blockquote>
<p>Yes, there are.</p>
<p>When Placement was extracted from Nova, one of the agreements the
new project team made was to pay greater attention to tech-debt
items, performance, and meaningful clean-ups. One of the reasons this
was possible was that by being extracted, Placement vastly limited
its scope and feature drive. Focused attention is easier and the
system is contained enough that unintended consequences from changes
are less frequent.</p>
<p>Another reason was that for several months my employer allowed me to
devote effectively 100% of my time to upstream work. That meant that
there was long term continuity of attention in my work. Minimal
feature work combined with maximal attention leads to some good
results.</p>
<p>In August I wrote up an analysis of some of that work in <a href="/placement-performance-analysis.html">Placement
Performance Analysis</a>,
explaining some of the things that were learned and changed. However
that analysis was comparing Placement code from the start of Train
to Train in August. I've since repeated some of the measurement,
comparing:</p>
<ol>
<li>Running Placement from the Nova codebase, using the <code>stable/stein</code>
branch.</li>
<li>Running Placement from the Placement codebase, using the
<code>stable/stein/</code> branch.</li>
<li>Running Placement from the Placement codebase, using <code>master</code>,
which at the moment is the same as what will become
<code>stable/train</code> and be released as <code>2.0.0</code>.</li>
</ol>
<p>The same database (PostgreSQL) and web server (uwsgi using four
processes of ten threads each) is used with each version of the
code. The database is pre-populated with 7000 resource providers
representing a suite of 1000 compute hosts with a moderately complex
nested provider topology that is similar to what might be used for a
virtualized network function.</p>
<p>The same query is used, whatever the latest microversion is for that
version:</p>
<div class="highlight"><pre><span></span>http://ds1:8000/allocation_candidates? \
resources=DISK_GB:10& \
required=COMPUTE_VOLUME_MULTI_ATTACH& \
resources1=VCPU:1,MEMORY_MB:256& \
required1=CUSTOM_FOO& \
resources2=FPGA:1& \
group_policy=none
</pre></div>
<p>(This is similar to what is used in the <code>nested-perfload</code> peformance
job in the testing gate, modified to work with all available
microversions.)</p>
<p>Here are some results, with some discussion after.</p>
<h3 id="10-serial-requests">10 Serial Requests</h3>
<p>Placement in Nova (stein)</p>
<div class="highlight"><pre><span></span>Requests per second: 0.06 [#/sec] (mean)
Time per request: 16918.522 [ms] (mean)
</pre></div>
<p>Extracted Placement (stein)</p>
<div class="highlight"><pre><span></span>Requests per second: 0.34 [#/sec] (mean)
Time per request: 2956.959 [ms] (mean)
</pre></div>
<p>Extracted Placement (train)</p>
<div class="highlight"><pre><span></span>Requests per second: 1.37 [#/sec] (mean)
Time per request: 730.566 [ms] (mean)
</pre></div>
<h3 id="100-requests-10-at-a-time">100 Requests, 10 at a time</h3>
<p>Placement in Nova (stein)</p>
<p><strong>This one failed</strong>. The numbers say:</p>
<div class="highlight"><pre><span></span>Requests per second: 0.18 [#/sec] (mean)
Time per request: 56567.575 [ms] (mean)
</pre></div>
<p>But of the 100 requests, 76 failed.</p>
<p>Extracted Placement (stein)</p>
<div class="highlight"><pre><span></span>Requests per second: 0.41 [#/sec] (mean)
Time per request: 24620.759 [ms] (mean)
</pre></div>
<p>Extracted Placement (train)</p>
<div class="highlight"><pre><span></span>Requests per second: 2.65 [#/sec] (mean)
Time per request: 3774.854 [ms] (mean)
</pre></div>
<p>The improvement between the versions in Stein (16.9s to 2.9s per
request) were mostly made through fairly obvious architecture and
code improvments found by inspection (or simply knowing it was not
ideal when first made, and finally getting around to fixing it).
Things like removing the use of <a href="https://docs.openstack.org/oslo.versionedobjects/latest/">oslo versioned
objects</a>
and changes to cache management to avoid redundant locks.</p>
<p>From Stein to Train (2.9s to .7s per request) the improvements were
made by doing detailed profiling and benchmarking and pursuing a
very active process of iteration (some of which is described by
<a href="/placement-performance-analysis.html">Placement Performance Analysis</a>).</p>
<p>In both cases this was possible because people (especially me) had
the "retained concentration" desired above by Kashyap. As a
community OpenStack needs to figure out how it can enshrine and
protect that attention and the associated experimentation and
consideration for long term health. I was able to do it in part
because I was able to get my employer to let me and in part because
I <a href="/eight-hour-day-update.html">overcommitted myself</a>.</p>
<p>Neither of these things are true any more. My employer has called me
inside, my upstream time will henceforth drop to "not much". I'm
optimistic that we've established a precedent and culture for doing
the right things in Placement, but it will be a challenge and I
don't think it is there in general for the whole community.</p>
<p>I've written about some of these things
<a href="/more-on-maintainership.html">before</a>. If the companies making
money off OpenStack are primarily focused on features (and being
disappointed when they can't get those features into Nova) who will
be focused on tech-debt, performance, and meaningful clean-ups? Who
will be aware of the systems well enough to effectively and
efficiently review all these proposed features? Who will clear up
tech-debt enough that the systems are easier to extend without
unintended consequences or risks?</p>
<p>Let's hit that Placement performance improvement some more, just to
make it clear:</p>
<p>In the tests above, "Placement in Nova (stein)" failed with a
concurrency of 10. I wanted to see at what concurrency "Extracted
Placement (train)" would fail: At 150 concurrency of 1000 requests
some requests fail. At 140 all requests work, albeit slow per
request (33s). Based on the error messages seen, the failing at 150
is tied to the sizing and configuration of the web server and
nothing to do with the placement code itself. The way to have higher
concurrency is to have more or larger web servers.</p>
<p>Remember that the nova version fails at concurrency of <strong>10</strong> with
the exact same web server setup. Find the time to fix your debt. It
will be worth it.</p>Placement Update 19-∞2019-09-27T14:32:00+01:002019-09-27T14:32:00+01:00Chris Denttag:anticdent.org,2019-09-27:/placement-update-19-.html<p>Let's call this placement update 19-∞, as this will be my last one.
It's been my pleasure to provide this service for nearly <a href="http://lists.openstack.org/pipermail/openstack-dev/2016-November/107171.html">three
years</a>.
I hope it has been as useful to others as it has been for me. The
goal all along was to provide some
<a href="https://en.wikipedia.org/wiki/Stigmergy">stigmergic</a> structures …</p><p>Let's call this placement update 19-∞, as this will be my last one.
It's been my pleasure to provide this service for nearly <a href="http://lists.openstack.org/pipermail/openstack-dev/2016-November/107171.html">three
years</a>.
I hope it has been as useful to others as it has been for me. The
goal all along was to provide some
<a href="https://en.wikipedia.org/wiki/Stigmergy">stigmergic</a> structures to
augment how we, the placement team, collaborated.</p>
<p>I guess it worked: yesterday we released a candidate that
will likely become the Train version of placement. The first version
where the only placement you can get is from its own repo/project.
Thanks to everyone who has made this possible over the years.</p>
<h1 id="most-important">Most Important</h1>
<p>Tetsuro will be the next placement PTL. He and Gibi are working on
the project update and other Summit/PTG-related activities for
Shanghai. If you have thoughts on that, please contact them.</p>
<p>The <a href="https://storyboard.openstack.org/#!/worklist/754">now worklist</a>
I made a couple weeks ago has had some progress, but some tasks
remain. None of them were critical for the release, except perhaps
the ongoing need for better documentation of how to most effectively
use the service.</p>
<p>Since we're expecting the Ussuri release to be one where we
consolidate how other services make use of placement, most of the
items on that list can fit well with that.</p>
<p>We should be on the lookout for bugs reported by people trying to
the release candidate(s).</p>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 21 (-2) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
7 (2) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 2 (-2)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 9
(-1) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
4 (-1) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 15 (-2).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 6 (0).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<p>There are
<a href="https://review.opendev.org/#/q/project:openstack/osc-placement+status:open">several</a>
osc-placement changes, many of which are related to the cutting of a
stable/train branch.</p>
<h1 id="main-themes">Main Themes</h1>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a>
No one had the time to address the fix ups for this before Train
was cut, so consumer types will be a part of Ussuri. If anyone is
available to adopt this, please let me and/or tssurya know.</li>
</ul>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we made
this cycle are fairly complex to use and were fairly complex to
write. We need to make sure, with the coming cycle, that we help
people use them well.</p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There are two <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And zero <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
changes</a>.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>Since we're in RC period I'll not bother listing pending changes.
During Ussuri the placement team hopes to be available to
facilitate other projects using the service. Keeping track of those
is what this section has been trying to do. Besides general
awareness of what's going on, I've also used <a href="https://review.opendev.org/#/q/(message:placement+OR+comment:placement)+status:open">this gerrit
query</a>
to find things that might be related to placement.</p>
<h1 id="end">End</h1>
<p>🙇</p>Placement Update 19-362019-09-13T12:18:00+01:002019-09-13T12:18:00+01:00Chris Denttag:anticdent.org,2019-09-13:/placement-update-19-36.html<p>Here's placement update 19-36. There won't be one next week, I will
be away. Because of my forthcoming "less time available for
OpenStack" I will also be stopping these updates at some point in
the next month or so so I can focus the limited time I will have on …</p><p>Here's placement update 19-36. There won't be one next week, I will
be away. Because of my forthcoming "less time available for
OpenStack" I will also be stopping these updates at some point in
the next month or so so I can focus the limited time I will have on
reviewing and coding. There will be at least one more.</p>
<h1 id="most-important">Most Important</h1>
<p>The big news this week is that after returning from a trip (that
meant he was away during the nomination period) Tetsuro has stepped
up to be the PTL for placement in Ussuri. Thanks very much to him
for taking this up, I'm sure he will be excellent.</p>
<p>We need to work on useful documentation for the features developed
this cycle.</p>
<p>I've also made a <a href="https://storyboard.openstack.org/#!/worklist/754">now
worklist</a> in
StoryBoard to draw attention to placement project stories that are
relevant to the next few weeks, making it easier to ignore those
that are not relevant now, but may be later.</p>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 23 (-1) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
5 (0) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 4 (0)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 10
(-1) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
5 (1) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 17 (0).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 6 (0).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<ul>
<li><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of. There's been some useful
discussion about how to achieve this, and a consensus has emerged
on how to get the best results.</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a>
This has some good comments on it from melwitt. I'm going to be
away next week, so if someone else would like to address them that
would be great. If it is deemed fit to merge, we should, despite
feature freeze passing, since we haven't had much churn lately. If
it doesn't make it in Train, that's fine too. The goal is to have
it ready for Nova in Ussuri as early as possible.</li>
</ul>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we are
making this cycle are fairly complex to use and are fairly complex
to write, so it is good that we're going to have plenty of time to
clean and clarify all these things.</p>
<p>Performance related explorations continue:</p>
<ul>
<li><a href="https://review.opendev.org/#/c/679385/">https://review.opendev.org/#/c/679385/</a>
Refactor initialization of research context. This puts the code
that might cause an exit earlier in the process so we can avoid
useless work.</li>
</ul>
<p>One outcome of the performance work needs to be something like a
<em>Deployment Considerations</em> document to help people choose how to
tweak their placement deployment to match their needs. The simple
answer is use more web servers and more database servers, but that's
often very wasteful.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/owner:"Chris+Dent+%253Ccdent%2540anticdent.org%253E"+topic:build-pdf-docs">https://review.opendev.org/#/q/owner:"Chris+Dent+%253Ccdent%2540anticdent.org%253E"+topic:build-pdf-docs</a>
These are the patches for meeting the build pdf docs goal for the
various placement projects.</li>
</ul>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There are three <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And two <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
changes</a>.
The latter are docs-related.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New reviews are added to the end of the list. Reviews that haven't
had attention in a long time (boo!) or have merged or approved
(yay!) are removed.</p>
<ul>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/670112">https://review.opendev.org/670112</a>
Nova: WIP: Add a placement audit command</p>
</li>
<li>
<p><a href="https://review.opendev.org/670696">https://review.opendev.org/670696</a>
tempest: Add placement API methods for testing routed provider nets</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/cross-cell-resize">https://review.opendev.org/#/q/topic:bp/cross-cell-resize</a>
Nova: cross cell resize</p>
</li>
<li>
<p><a href="https://review.opendev.org/674524">https://review.opendev.org/674524</a>
Nova: Scheduler translate properties to traits</p>
</li>
<li>
<p><a href="https://review.opendev.org/623558">https://review.opendev.org/623558</a>
Nova: single pass instance info fetch in host manager</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/provider-config-file">https://review.opendev.org/#/q/topic:bp/provider-config-file</a>
Nova: using provider config file for custom resource providers</p>
</li>
<li>
<p><a href="https://review.opendev.org/681955">https://review.opendev.org/681955</a>
Nova: clean up some lingering placement stuff</p>
</li>
<li>
<p><a href="https://review.opendev.org/664867">https://review.opendev.org/664867</a>
OSA: Add nova placement to placement migration</p>
</li>
<li>
<p><a href="https://review.opendev.org/681343">https://review.opendev.org/681343</a>
Charms: Disable nova placement API in Train</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bug/1841481">https://review.opendev.org/#/q/topic:bug/1841481</a>
Nova: stop using @safe_connect in report client</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>🐈</p>Placement Update 19-352019-09-06T11:53:00+01:002019-09-06T11:53:00+01:00Chris Denttag:anticdent.org,2019-09-06:/placement-update-19-35.html<p>Let's have a placement update 19-35. Feature freeze is this week. We
have a feature in progress (consumer types, see below) but it is not
critical.</p>
<h1 id="most-important">Most Important</h1>
<p>Three main things we should probably concern ourselves with in the
immediate future:</p>
<ul>
<li>
<p>We are currently without a PTL for Ussuri. There's …</p></li></ul><p>Let's have a placement update 19-35. Feature freeze is this week. We
have a feature in progress (consumer types, see below) but it is not
critical.</p>
<h1 id="most-important">Most Important</h1>
<p>Three main things we should probably concern ourselves with in the
immediate future:</p>
<ul>
<li>
<p>We are currently without a PTL for Ussuri. There's some discussion
about the options for dealing with this in an <a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-September/thread.html#9131">email
thread</a>.
If you have ideas (or want to put yourself forward), please share.</p>
</li>
<li>
<p>We need to work on useful documentation for the features developed
this cycle.</p>
</li>
<li>
<p>We need to create some <a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009137.html">cycle
highlights</a>.
To help with that I've started <a href="https://etherpad.openstack.org/p/placement-train-cycle-highlights">an
etherpad</a>.
If I've forgotten anything, please make additions.</p>
</li>
</ul>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p>osc-placement 1.7.0 has been
<a href="https://pypi.org/project/osc-placement/">released</a>. This adds
support for <a href="https://review.opendev.org/#/q/topic:allocation-ratios+(status:open+OR+status:merged)">managing allocation
ratios</a>
via aggregates, but adding a few different commands and args for
inventory manipulation.</p>
</li>
<li>
<p>Work on consumer types exposed that placement needed to be first
class in grenade to make sure database migrations are run. That
<a href="https://review.opendev.org/679655">change has merged</a>. Until then
placement was upgraded as part of nova.</p>
</li>
</ul>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 24 (-1) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
5 (0) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 4 (0)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 11
(-1) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
4 (0) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 17 (0).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 6 (0).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<ul>
<li>
<p><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of. There's been some useful
discussion about how to achieve this, and a consensus has emerged
on how to get the best results.</p>
</li>
<li>
<p><code>--amend</code> and <code>--aggregate</code> on resource provider inventory has
merged and been release 1.7.0 (see above).</p>
</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a>
I took this through to microversion and api-ref docs, so it is
ready for wider review. If this doesn't make it in for Train,
that's okay. The goal is to have it ready for Nova to start
working with it when Nova is able.</li>
</ul>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we are
making this cycle are fairly complex to use and are fairly complex
to write, so it is good that we're going to have plenty of time to
clean and clarify all these things.</p>
<p>Performance related explorations continue:</p>
<ul>
<li><a href="https://review.opendev.org/#/c/679385/">https://review.opendev.org/#/c/679385/</a>
Refactor initialization of research context. This puts the code
that might cause an exit earlier in the process so we can avoid
useless work.</li>
</ul>
<p>One outcome of the performance work needs to be something like a
<em>Deployment Considerations</em> document to help people choose how to
tweak their placement deployment to match their needs. The simple
answer is use more web servers and more database servers, but that's
often very wasteful.</p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<ul>
<li><a href="https://review.opendev.org/676982">https://review.opendev.org/676982</a>
Merge request log and request id middlewares is worth attention.
It makes sure that <em>all</em> log message from a single request use a
global and local request id.</li>
</ul>
<p>There are three <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And zero <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
changes</a>.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>This week (because of feature freeze) I will not be adding new finds
to the list, just updating what was already on the list.</p>
<ul>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/virtual-persistent-memory">https://review.opendev.org/#/q/topic:bp/virtual-persistent-memory</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/670112">https://review.opendev.org/670112</a>
Nova: WIP: Add a placement audit command</p>
</li>
<li>
<p><a href="https://review.opendev.org/671793">https://review.opendev.org/671793</a>
Nova: libvirt: Start reporting PCPU inventory to placement
A part of <a href="https://review.opendev.org/#/q/topic:bp/cpu-resources">https://review.opendev.org/#/q/topic:bp/cpu-resources</a></p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/667952">https://review.opendev.org/667952</a>
nova: Support filtering of hosts by forbidden aggregates</p>
</li>
<li>
<p><a href="https://review.opendev.org/670696">https://review.opendev.org/670696</a>
tempest: Add placement API methods for testing routed provider nets</p>
</li>
<li>
<p><a href="https://review.opendev.org/672678">https://review.opendev.org/672678</a>
openstack-helm: Build placement in OSH-images</p>
</li>
<li>
<p><a href="https://review.opendev.org/674129">https://review.opendev.org/674129</a>
Correct global_request_id sent to Placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/cross-cell-resize">https://review.opendev.org/#/q/topic:bp/cross-cell-resize</a>
Nova: cross cell resize</p>
</li>
<li>
<p><a href="https://review.opendev.org/674524">https://review.opendev.org/674524</a>
Nova: Scheduler translate properties to traits</p>
</li>
<li>
<p><a href="https://review.opendev.org/623558">https://review.opendev.org/623558</a>
Nova: single pass instance info fetch in host manager</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/provider-config-file">https://review.opendev.org/#/q/topic:bp/provider-config-file</a>
Nova: using provider config file for custom resource providers</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>🐎</p>Placement Update 19-342019-08-30T10:48:00+01:002019-08-30T10:48:00+01:00Chris Denttag:anticdent.org,2019-08-30:/placement-update-19-34.html<p>Welcome to placement update 19-34. Feature Freeze is the week of
September 9th. We have features in progress in placement itself
(consumer types) and osc-placement that would be great to land.</p>
<h1 id="most-important">Most Important</h1>
<p>In addition to the features above, we really need to get started on
tuning up the documentation …</p><p>Welcome to placement update 19-34. Feature Freeze is the week of
September 9th. We have features in progress in placement itself
(consumer types) and osc-placement that would be great to land.</p>
<h1 id="most-important">Most Important</h1>
<p>In addition to the features above, we really need to get started on
tuning up the documentation so that <code>same_subtree</code> and friends can
be used effectively.</p>
<p>It is also time to start thinking about what features, if any, need
to be pursued in Ussuri. If there are few, that ought to leave time
and energy for getting the osc-placement plugin more up to date.</p>
<p>And, there are plenty of stories (see below) that need attention.
Ideally we'd end every cycle with zero stories, including removing
ones that no longer make sense.</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>Tetsuro has picked up the baton for performance and refactoring
work and found <a href="https://review.opendev.org/677120">some</a>
<a href="https://review.opendev.org/677209">improvements</a> that have
merged. There's additional work in progress (noted below).</li>
</ul>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 25 (2) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
5 (1) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 4 (0)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 12
(1) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
4 (0) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 17 (-1).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 6 (2).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<p>osc-placement is currently behind by 12 microversions.</p>
<ul>
<li>
<p><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of. There's been some useful
discussion about how to achieve this, and a consensus has emerged
on how to get the best results.</p>
</li>
<li>
<p><a href="https://review.opendev.org/640898">https://review.opendev.org/640898</a>
Adds a new <code>--amend</code> option which can update resource provider
inventory without requiring the user to pass a full replacement
for inventory and an <code>--aggregate</code> option to set inventory on all
the providers in an aggregate. This has been broken up into three
patches to help with review. This one is very close but needs
review from more people than Matt.</p>
</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a>
A WIP, as microversion 1.37, has started.</li>
</ul>
<p>I picked this up yesterday and hope to have it finished next week,
barring distractions. I figure having it in place for nova for
Ussuri is a nice to have.</p>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we are
making this cycle are fairly complex to use and are fairly complex
to write, so it is good that we're going to have plenty of time to
clean and clarify all these things.</p>
<p>Performance related explorations continue:</p>
<ul>
<li><a href="https://review.opendev.org/#/c/679385/">https://review.opendev.org/#/c/679385/</a>
Refactor initialization of research context. This puts the code
that might cause an exit earlier in the process so we can avoid
useless work.</li>
</ul>
<p>One outcome of the performance work needs to be something like a
<em>Deployment Considerations</em> document to help people choose how to
tweak their placement deployment to match their needs. The simple
answer is use more web servers and more database servers, but that's
often very wasteful.</p>
<p>Discussions about <a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-August/thread.html#8849">using a different JSON
serializer</a>
terminated choosing not to use <code>orjson</code> because it presents some
packaging and distribution issues that might be problematic. There's
still an option to use one of the other alternatives, but that
exploration has not started.</p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<ul>
<li><a href="https://review.opendev.org/676982">https://review.opendev.org/676982</a>
Merge request log and request id middlewares is worth attention.
It makes sure that <em>all</em> log message from a single request use a
global and local request id.</li>
</ul>
<p>There are two <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And zero <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
changes</a>.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New discoveries are added to the end. Merged stuff is removed.
Anything that has had no activity in 4 weeks has been removed.</p>
<ul>
<li>
<p><a href="https://review.opendev.org/659233">https://review.opendev.org/659233</a>
Cyborg: Placement report</p>
</li>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/virtual-persistent-memory">https://review.opendev.org/#/q/topic:bp/virtual-persistent-memory</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/670112">https://review.opendev.org/670112</a>
Nova: WIP: Add a placement audit command</p>
</li>
<li>
<p><a href="https://review.opendev.org/671793">https://review.opendev.org/671793</a>
Nova: libvirt: Start reporting PCPU inventory to placement
A part of <a href="https://review.opendev.org/#/q/topic:bp/cpu-resources">https://review.opendev.org/#/q/topic:bp/cpu-resources</a></p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/666202">https://review.opendev.org/666202</a>
Blazar: Create placement client for each request</p>
</li>
<li>
<p><a href="https://review.opendev.org/667952">https://review.opendev.org/667952</a>
nova: Support filtering of hosts by forbidden aggregates</p>
</li>
<li>
<p><a href="https://review.opendev.org/669079">https://review.opendev.org/669079</a>
blazar: Send global_request_id for tracing calls</p>
</li>
<li>
<p><a href="https://review.opendev.org/670696">https://review.opendev.org/670696</a>
tempest: Add placement API methods for testing routed provider nets</p>
</li>
<li>
<p><a href="https://review.opendev.org/672678">https://review.opendev.org/672678</a>
openstack-helm: Build placement in OSH-images</p>
</li>
<li>
<p><a href="https://review.opendev.org/674129">https://review.opendev.org/674129</a>
Correct global_request_id sent to Placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/cross-cell-resize">https://review.opendev.org/#/q/topic:bp/cross-cell-resize</a>
Nova: cross cell resize</p>
</li>
<li>
<p><a href="https://review.opendev.org/674524">https://review.opendev.org/674524</a>
Nova: Scheduler translate properties to traits</p>
</li>
<li>
<p><a href="https://review.opendev.org/623558">https://review.opendev.org/623558</a>
Nova: single pass instance info fetch in host manager</p>
</li>
<li>
<p><a href="https://review.opendev.org/674708">https://review.opendev.org/674708</a>
Zun: [WIP] Claim container allocation in placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/provider-config-file">https://review.opendev.org/#/q/topic:bp/provider-config-file</a>
Nova: using provider config file for custom resource providers</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>☃</p>Placement Update 19-322019-08-16T15:34:00+01:002019-08-16T15:34:00+01:00Chris Denttag:anticdent.org,2019-08-16:/placement-update-19-32.html<p>Here's placement update 19-32. There will be no update 33; I'm going
to take next week off. If there are Placement-related issues that
need immediate attention please speak with any of Eric Fried
(efried), Balazs Gibizer (gibi), or Tetsuro Nakamura (tetsuro).</p>
<h1 id="most-important">Most Important</h1>
<p>Same as last week: The main things …</p><p>Here's placement update 19-32. There will be no update 33; I'm going
to take next week off. If there are Placement-related issues that
need immediate attention please speak with any of Eric Fried
(efried), Balazs Gibizer (gibi), or Tetsuro Nakamura (tetsuro).</p>
<h1 id="most-important">Most Important</h1>
<p>Same as last week: The main things on the Placement radar are
implementing Consumer Types and cleanups, performance analysis, and
documentation related to nested resource providers.</p>
<p>A thing we should place on the "important" list is bringing the osc
placement plugin up to date. We also need to discuss what would we
would like the plugin to be. Is it required that it have ways to
perform all the functionality of the API, or is it about providing
ways to do what humans need to do with the placement API? Is there a
difference?</p>
<p>We decided that consumer types is medium priority: The nova-side use
of the functionality is not going to happen in Train, but it would
be nice to have the placement-side ready when U opens. The primary
person working on it, tssurya, is spread pretty thin so it might not
happen unless someone else has the cycles to give it some attention.</p>
<p>On the documentation front, we realized during some performance work
<a href="https://review.opendev.org/675606">last</a>
<a href="https://review.opendev.org/#/c/676204/4/placement/tests/functional/gabbits/same-subtree-deep.yaml@29">week</a>
that it easy to have an incorrect grasp of how <code>same_subtree</code> works
when there are more than two groups involved. It is critical that we
create good "how to use" documentation for this and other advanced
placement features. Not only can it be easy to get wrong, it can be
challenge to see that you've got it wrong (the failure mode is "more
results, only some of which you actually wanted").</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p>Yet more <a href="https://review.opendev.org/#/q/topic:optimize-_build_provider_summaries">performance
fixes</a>
are in the process of merging. Most of these are related to
getting <code>_merge_candidates</code> and <code>_build_provider_summaries</code> to
have less impact. The fixes are generally associated with avoiding
duplicate work by generating dicts of reusable objects earlier in
the request. This is possible because of the relatively new
<code>RequestWideSearchContext</code>. In a request that returns many
provider summaries <code>_build_provider_summaries</code> continues to have a
significant impact because it has to create many objects but
overall everything is much less heavyweight. More on performance
in Themes, below.</p>
</li>
<li>
<p>The combination of all these performance fixes, and because of
microversions, makes it reasonable for anyone running placement in
a resource constrained environment (or simply wanting things to be
faster) to consider running Train placement with <em>any</em> release of
OpenStack. Obviously you should test it first, but it is worth
investigating. More information on how to achieve this can be
found in the <a href="https://docs.openstack.org/placement/latest/upgrade/to-stein.html">upgrade to stein
docs</a></p>
</li>
</ul>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 23 (1) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
4 (1) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 4 (0)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 11
(0) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
4 (0) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 18 (1).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 4 (-1).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<p>osc-placement is currently behind by 12 microversions.</p>
<ul>
<li>
<p><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of. There's been some useful
discussion about how to achieve this, and a consensus has emerged
on how to get the best results.</p>
</li>
<li>
<p><a href="https://review.opendev.org/640898">https://review.opendev.org/640898</a>
Adds a new '--amend' option which can update resource provider
inventory without requiring the user to pass a full replacement
for inventory. This has been broken up into three patches to help
with review.</p>
</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a>
A WIP, as microversion 1.37, has started.</li>
</ul>
<p>As mentioned above, this is currently paused while other things take
priority. If you have time that you could spend on this please
respond here expressing that interest.</p>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we are
making this cycle are fairly complex to use and are fairly complex
to write, so it is good that we're going to have plenty of time to
clean and clarify all these things.</p>
<p>As said above, there's lots of performance work in progress. We'll
need to make a similar effort with regard to docs. For example, all
of the coders involved in the creation and review of the
<code>same_subtree</code> functionality struggle to explain, clearly and
simply, how it will work in a variety of situations. We need to
enumerate the situations and the outcomes, in documentation.</p>
<p>One outcome of this work will be something like a <em>Deployment
Considerations</em> document to help people choose how to tweak their
placement deployment to match their needs. The simple answer is use
more web servers and more database servers, but that's often very
wasteful.</p>
<p>On the performance front, there is one major area of impact which
has not received much attention yet. When requesting allocation
candidates (or resource providers) that will return many results
the cost of JSON serialization is just under one quarter of the
processing time. This is to be expected when the response body is
<code>2379k</code> big, and 154000 lines long (when pretty printed) for 7000
provider summaries and 2000 allocation requests.</p>
<p>But there are ways to fix it. One is to ask more focused questions
(so fewer results are expected). Another is to <code>limit=N</code> the results
(but this can lead to issues with migrations).</p>
<p>Another is to <a href="https://review.opendev.org/674661">use a different JSON
serializer</a>. Should we do that?
It make a <em>big</em> difference with large result sets (which will be
common in big and sparse clouds).</p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There are two <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And zero <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
changes</a>.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New discoveries are added to the end. Merged stuff is removed.
Anything that has had no activity in 4 weeks has been removed.</p>
<ul>
<li>
<p><a href="https://review.openstack.org/#/q/topic:bug/1819923">https://review.openstack.org/#/q/topic:bug/1819923</a>
Nova: nova-manage: heal port allocations</p>
</li>
<li>
<p><a href="https://review.opendev.org/659233">https://review.opendev.org/659233</a>
Cyborg: Placement report</p>
</li>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/634551">https://review.opendev.org/634551</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/670112">https://review.opendev.org/670112</a>
Nova: WIP: Add a placement audit command</p>
</li>
<li>
<p><a href="https://review.opendev.org/671312">https://review.opendev.org/671312</a>
blazar: Fix placement operations in multi-region deployments</p>
</li>
<li>
<p><a href="https://review.opendev.org/671793">https://review.opendev.org/671793</a>
Nova: libvirt: Start reporting PCPU inventory to placement
A part of <https://review.opendev.org/#/q/topic:bp/cpu-resources</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/666202">https://review.opendev.org/666202</a>
Blazar: Create placement client for each request</p>
</li>
<li>
<p><a href="https://review.opendev.org/667952">https://review.opendev.org/667952</a>
nova: Support filtering of hosts by forbidden aggregates</p>
</li>
<li>
<p><a href="https://review.opendev.org/669079">https://review.opendev.org/669079</a>
blazar: Send global_request_id for tracing calls</p>
</li>
<li>
<p><a href="https://review.opendev.org/670696">https://review.opendev.org/670696</a>
tempest: Add placement API methods for testing routed provider nets</p>
</li>
<li>
<p><a href="https://review.opendev.org/672678">https://review.opendev.org/672678</a>
openstack-helm: Build placement in OSH-images</p>
</li>
<li>
<p><a href="https://review.opendev.org/674129">https://review.opendev.org/674129</a>
Correct global_request_id sent to Placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/cross-cell-resize">https://review.opendev.org/#/q/topic:bp/cross-cell-resize</a>
Nova: cross cell resize</p>
</li>
<li>
<p><a href="https://review.opendev.org/674524">https://review.opendev.org/674524</a>
Nova: Scheduler translate properties to traits</p>
</li>
<li>
<p><a href="https://review.opendev.org/623558">https://review.opendev.org/623558</a>
Nova: single pass instance info fetch in host manager</p>
</li>
<li>
<p><a href="https://review.opendev.org/674708">https://review.opendev.org/674708</a>
Zun: [WIP] Claim container allocation in placement</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>Have a good next week.</p>Placement Update 19-312019-08-09T15:07:00+01:002019-08-09T15:07:00+01:00Chris Denttag:anticdent.org,2019-08-09:/placement-update-19-31.html<p>Pupdate 19-31. No bromides today.</p>
<h1 id="most-important">Most Important</h1>
<p>Same as last week: The main things on the Placement radar are
implementing Consumer Types and cleanups, performance analysis, and
documentation related to nested resource providers.</p>
<p>We need to decide how much of a priority consumer types support is.
I've taken the task …</p><p>Pupdate 19-31. No bromides today.</p>
<h1 id="most-important">Most Important</h1>
<p>Same as last week: The main things on the Placement radar are
implementing Consumer Types and cleanups, performance analysis, and
documentation related to nested resource providers.</p>
<p>We need to decide how much of a priority consumer types support is.
I've taken the task of asking around with the various interested
parties.</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p>A more complex nested topology is now being used in the
nested-perfload check job, and both that and the non-nest perfload
run <a href="https://review.opendev.org/#/c/673540/">apache benchmark</a> at
the end. When you make changes you can have a look at the results
of the <code>placement-perfload</code> and <code>placement-nested-perfload</code> gate
jobs to see if there has been a performance impact. Keep in mind
the numbers are only a guide. The performance characteristics of
VMs from different CI providers varies <em>wildly</em>.</p>
</li>
<li>
<p>A stack of several performance related improvements has merged,
with still more to come. I've written a separate <a href="https://anticdent.org/placement-performance-analysis.html">Placement
Performance
Analysis</a>
that summarizes some of the changes. Many of these may be useful
for other services. Each iteration reveals another opportunity.</p>
</li>
<li>
<p>In some environments placement will receive a URL of '' when '/'
is expected. Auth handling for version control needs to <a href="https://review.opendev.org/674543">handle
this</a>.</p>
</li>
<li>
<p>osc-placmeent 1.6.0 is in the process of being
<a href="https://review.opendev.org/675311">released</a>.</p>
</li>
</ul>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 22 (-1) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
3 (0) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 4 (-1)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 11
(0) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
4 (0) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 17 (0).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 5 (1).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<p>osc-placement is currently behind by 12 microversions.</p>
<ul>
<li>
<p><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of. There's been some useful
discussion about how to achieve this, and a consensus has emerged
on how to get the best results.</p>
</li>
<li>
<p><a href="https://review.opendev.org/640898">https://review.opendev.org/640898</a>
Adds a new '--amend' option which can update resource provider
inventory without requiring the user to pass a full replacement
for inventory. This has been broken up into three patches to help
with review.</p>
</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a>
A WIP, as microversion 1.37, has started.</li>
</ul>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we are
making this cycle are fairly complex to use and are fairly complex
to write, so it is good that we're going to have plenty of time to
clean and clarify all these things.</p>
<p>As said above, there's lots of performance work in progress. We'll
need to make a similar effort with regard to docs.</p>
<p>One outcome of this work will be something like a <em>Deployment
Considerations</em> document to help people choose how to tweak their
placement deployment to match their needs. The simple answer is use
more web servers and more database servers, but that's often very
wasteful.</p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There are two <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And zero <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
changes</a>.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New discoveries are added to the end. Merged stuff is removed.
Anything that has had no activity in 4 weeks has been removed.</p>
<ul>
<li>
<p><a href="https://review.openstack.org/#/q/topic:bug/1819923">https://review.openstack.org/#/q/topic:bug/1819923</a>
Nova: nova-manage: heal port allocations</p>
</li>
<li>
<p><a href="https://review.opendev.org/659233">https://review.opendev.org/659233</a>
Cyborg: Placement report</p>
</li>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/634551">https://review.opendev.org/634551</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/670112">https://review.opendev.org/670112</a>
Nova: WIP: Add a placement audit command</p>
</li>
<li>
<p><a href="https://review.opendev.org/671312">https://review.opendev.org/671312</a>
blazar: Fix placement operations in multi-region deployments</p>
</li>
<li>
<p><a href="https://review.opendev.org/671793">https://review.opendev.org/671793</a>
Nova: libvirt: Start reporting PCPU inventory to placement
A part of <https://review.opendev.org/#/q/topic:bp/cpu-resources</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/666202">https://review.opendev.org/666202</a>
Blazar: Create placement client for each request</p>
</li>
<li>
<p><a href="https://review.opendev.org/667952">https://review.opendev.org/667952</a>
nova: Support filtering of hosts by forbidden aggregates</p>
</li>
<li>
<p><a href="https://review.opendev.org/669079">https://review.opendev.org/669079</a>
blazar: Send global_request_id for tracing calls</p>
</li>
<li>
<p><a href="https://review.opendev.org/668252">https://review.opendev.org/668252</a>
Nova: Update HostState.*_allocation_ratio earlier</p>
</li>
<li>
<p><a href="https://review.opendev.org/670696">https://review.opendev.org/670696</a>
tempest: Add placement API methods for testing routed provider nets</p>
</li>
<li>
<p><a href="https://review.opendev.org/672678">https://review.opendev.org/672678</a>
openstack-helm: Build placement in OSH-images</p>
</li>
<li>
<p><a href="https://review.opendev.org/674129">https://review.opendev.org/674129</a>
Correct global_request_id sent to Placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/cross-cell-resize">https://review.opendev.org/#/q/topic:bp/cross-cell-resize</a>
Nova: cross cell resize</p>
</li>
<li>
<p><a href="https://review.opendev.org/675506">https://review.opendev.org/675506</a>
Watcher: Remove resource used fields from ComputeNode</p>
</li>
<li>
<p><a href="https://review.opendev.org/674524">https://review.opendev.org/674524</a>
Nova: Scheduler translate properties to traits</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>Somewhere in this performance work is a lesson for life: Every time
I think we've reached the bottom of the "easy stuff", I find yet
another bit of easy stuff.</p>Placement Performance Analysis2019-08-08T17:30:00+01:002019-08-08T17:30:00+01:00Chris Denttag:anticdent.org,2019-08-08:/placement-performance-analysis.html<p>Performance has always been important to <a href="https://docs.openstack.org/placement/latest/">Placement</a>.
In a busy <a href="https://www.openstack.org/">OpenStack</a> cloud, it will
receive many hits per second. Any slowness in the placement service
will add to the latency present in instance creation and migration
operations.</p>
<p>When we added support for requesting complex topologies of nested
resource providers, performance …</p><p>Performance has always been important to <a href="https://docs.openstack.org/placement/latest/">Placement</a>.
In a busy <a href="https://www.openstack.org/">OpenStack</a> cloud, it will
receive many hits per second. Any slowness in the placement service
will add to the latency present in instance creation and migration
operations.</p>
<p>When we added support for requesting complex topologies of nested
resource providers, performance took an expected hit. All along, the
plan was to make it work and then make it fast. In the last few
weeks members of the Placement team have been working to improve
performance.</p>
<p>Human analysis of the code can sometimes suggest obvious areas for
performance improvement but it is also very easy to be misled. It's
better to use profiling and benchmarking to get accurate
measurements of what code is using the most CPU and to effectively
compare different revisions of the code.</p>
<p>I've written <a href="/profiling-wsgi-apps.html">two</a>
<a href="/profiling-placement-in-docker.html">other</a> postings about
how to profile WSGI apps and analyse the results. Using those
strategies we've iterated through a series of changes using the
following process:</p>
<ol>
<li>profile to find the most expensive chunk of code</li>
<li>determine if it can be improved and how</li>
<li>change the code</li>
<li>benchmark to see if it really helps, if it does, keep it,
otherwise try something else</li>
<li>repeat</li>
</ol>
<p>The most recent big feature added to placement was called
<a href="https://review.opendev.org/668376">same_subtree</a>. It adds support
for requiring that a subset of the solution set for a request be
under the same ancestor resource provider. This helps to support
"affinity" within a compute host (e.g., "<em>this</em> FPGA is under the
same NUMA node as <em>this</em> FPGA").</p>
<p>What follows are some comparison numbers from benchmarks run with
the commit that added <code>same_subtree</code> and recent master (between
which several performance tweaks have been added). The test host is a
Linux VM with 16 GB of RAM, 16 VCPU. Placement is running standalone
(without keystone), using PostgreSQL as its database and uwsgi as
the web server with the following startup</p>
<div class="highlight"><pre><span></span>uwsgi --http :8000 --wsgi-file .tox/py37/bin/placement-api --processes 4 --threads 10
</pre></div>
<p>all on that same host.</p>
<p><a href="https://httpd.apache.org/docs/2.4/programs/ab.html">Apache
benchmark</a> is
run on an otherwise idle 8 core machine on the same local network.
Headers are set with <code>-H 'x-auth-token: admin'</code> and
<code>-H 'openstack-api-version: placement latest'</code> to drive appropriate
<code>noauth2</code> and microversion settings.</p>
<p>The server is preloaded with 7000 resource providers created using
the <a href="https://opendev.org/openstack/placement/src/branch/master/gate/gabbits/nested-perfload.yaml">nested-perfload
topology</a>.</p>
<p>The URL requested is:</p>
<div class="highlight"><pre><span></span><span class="err">GET /allocation_candidates?</span>
<span class="err"> resources=DISK_GB:10&</span>
<span class="err"> required=COMPUTE_VOLUME_MULTI_ATTACH&</span>
<span class="err"> resources_COMPUTE=VCPU:1,MEMORY_MB:256&</span>
<span class="err"> required_COMPUTE=CUSTOM_FOO&</span>
<span class="err"> resources_FPGA=FPGA:1&</span>
<span class="err"> group_policy=none&</span>
<span class="err"> same_subtree=_COMPUTE,_FPGA</span>
</pre></div>
<h2 id="the-older-code">The Older Code</h2>
<p><strong><code>ab -c 1 -n 10 [the rest]</code></strong> (1 concurrency, 10 total requests):</p>
<div class="highlight"><pre><span></span>Requests per second: 0.40 [#/sec] (mean)
Time per request: 2472.930 [ms] (mean)
</pre></div>
<p><strong><code>ab -c 40 -n 400 [the rest]</code></strong> (40 concurrency, 400 total requests):</p>
<div class="highlight"><pre><span></span>Requests per second: 1.46 [#/sec] (mean)
Time per request: 27454.696 [ms] (mean)
</pre></div>
<p>(For concerned benchmark purists: throughout this process I've also
been running with thousands of requests instead of tens or hundreds
to make sure that the mean values I'm getting here aren't because of
the short run time. They are not. Also, not reported here, but I've
also been doing benchmarks to compare how concurrent I can get
before something explodes. As you might expect: as individual
requests become lighter, the wider we can get.)</p>
<h2 id="the-new-and-improved-code">The New and Improved Code</h2>
<p>(These numbers are not quite up to date. They are from a recent
master but there are at least four more performance-related patches
yet to merge. I'll update when that's all in.)</p>
<p><strong><code>ab -c 1 -n 10 [the rest]</code></strong> (1 concurrency, 10 total requests):</p>
<div class="highlight"><pre><span></span>Requests per second: 0.70 [#/sec] (mean)
Time per request: 1423.695 [ms] (mean)
</pre></div>
<p><strong><code>ab -c 40 -n 400 [the rest]</code></strong> (40 concurrency, 400 total requests):</p>
<div class="highlight"><pre><span></span>Requests per second: 2.90 [#/sec] (mean)
Time per request: 13772.054 [ms] (mean)
</pre></div>
<h2 id="howd-we-get-there">How'd We Get There?</h2>
<p>This is a nice improvement. It may not seem like that much — over
1 second per request is rather slow in the absolute — but there is a
lot happening in the background and a lot of data being returned.</p>
<p><strong>One response is a complex nested JSON object of <code>2583330</code> bytes. It
has <code>154006</code> lines when sent through <code>json_pp</code>.</strong></p>
<p>There are several classes of changes that were made. These might be
applicable to other environments (like yours!):</p>
<ul>
<li>
<p>If using SQLAlchemy, using the
<a href="https://docs.sqlalchemy.org/en/13/core/connections.html?highlight=rowproxy#sqlalchemy.engine.RowProxy">RowProxy</a>
object directly, within the persistence layer, is okay and much
faster that casting to a <code>dict</code> or <code>namedtuple</code> (which have
interfaces the <code>RowProxy</code> already provides).</p>
</li>
<li>
<p>Use
<a href="https://docs.python.org/3/reference/datamodel.html#slots">__slots__</a>
in frequently used objects. It really does speed up attribute
access time.</p>
</li>
<li>
<p>Profiling can often reveal sets of data that are retrieved
multiple times. If you can find these and build them incrementally
in the context of a single request/operation it can be a big win.
See <a href="https://review.opendev.org/674254">Add RequestWideSearchContext.summaries_by_id</a>
and <a href="https://review.opendev.org/674581">Track usage info on RequestWideSearchContext</a>
for examples.</p>
</li>
<li>
<p>If you're doing anything with membership checking with a <code>list</code>
and you're able to make it a <code>set</code>, do.</p>
</li>
<li>
<p>When using SQLAlchemy's
<a href="https://docs.sqlalchemy.org/en/13/core/sqlelement.html?highlight=in_#sqlalchemy.sql.operators.ColumnOperators.in_">in_</a>
operator with a large number of values, an <a href="https://docs.sqlalchemy.org/en/13/core/sqlelement.html?highlight=in_#sqlalchemy.sql.expression.bindparam">expanding
bindparam</a>
can make a big difference in performance.</p>
</li>
<li>
<p>Implementing <code>__copy__</code> on simple classes of object that are
copied many times in single requests. Python's naive copy is
expensive, in aggregate.</p>
</li>
</ul>
<p>Also, not based on the recent profiling, but in earlier work
comparing non-nested setups (we've gone from <strong>1.2 seconds</strong> for a <code>GET
/allocation_candidates?resources=DISK_GB:10,VCPU:1,MEMORY_MB:256</code>
request against 1000 providers in early January to <strong>.53 seconds</strong> now)
we learned the following:</p>
<ul>
<li>Unless you absolutely must (perhaps because you are doing RPC),
avoid using <a href="https://docs.openstack.org/oslo.versionedobjects/latest/">oslo versioned
objects</a>.
They add a <em>lot</em> of overhead for type checking and coercing when
getting and setting attributes.</li>
</ul>
<h2 id="whats-next">What's Next?</h2>
<p>I'm pretty sure there are a lot more improvements to be made. Each
pass through the steps listed above exposes another avenue for
investigation. Thus far we've been able to make improvements without
too much awareness of the incoming request: we've not been adding
conditionals or special-cases. Adding those will probably take us
into a new world of opportunities.</p>
<p>Most of the application time is spent interacting with the
database. Little has yet been done to explore tweaking the schema
(things like de-normalization) or tweaking the database configuration
(threads available, cache sizes, using SSDs). All of that will have
impact.</p>
<p>And, in the end, because Placement is a simple web application over
a database, the easiest way to get more performance is to make more
web and database servers and load balance them. However, that's a
cop out, we should save cycles where we can. Everything is expensive
at scale.</p>Placement Update 19-302019-08-02T13:35:00+01:002019-08-02T13:35:00+01:00Chris Denttag:anticdent.org,2019-08-02:/placement-update-19-30.html<p>Pupdate 19-30 is brought to you by the letter P for Performance.</p>
<h1 id="most-important">Most Important</h1>
<p>The main things on the Placement radar are implementing Consumer
Types and cleanups, performance analysis, and documentation related
to nested resource providers.</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p><a href="https://review.opendev.org/673294">os-traits 0.16.0 was released</a>
was released, with a corresponding <a href="https://review.opendev.org/673964">canary …</a></p></li></ul><p>Pupdate 19-30 is brought to you by the letter P for Performance.</p>
<h1 id="most-important">Most Important</h1>
<p>The main things on the Placement radar are implementing Consumer
Types and cleanups, performance analysis, and documentation related
to nested resource providers.</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p><a href="https://review.opendev.org/673294">os-traits 0.16.0 was released</a>
was released, with a corresponding <a href="https://review.opendev.org/673964">canary
update</a>.</p>
</li>
<li>
<p>The <a href="https://review.opendev.org/673788">ProviderIds namedtuple was
removed</a>. This is the first in
a series of performance optimizations discovered as part of the
analysis described below in the Cleanup section.</p>
</li>
</ul>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 23 (2) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
3 (1) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 5 (0)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 11
(1) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
4 (0) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 17 (0).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 4 (0).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<p>osc-placement is currently behind by 12 microversions.</p>
<ul>
<li>
<p><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of. There's been some useful
discussion about how to achieve this, and a consensus has emerged
on how to get the best results.</p>
</li>
<li>
<p><a href="https://review.opendev.org/640898">https://review.opendev.org/640898</a>
Adds a new '--amend' option which can update resource provider
inventory without requiring the user to pass a full replacement
for inventory</p>
</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a>
A WIP, as microversion 1.37, has started.</li>
</ul>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we are
making this cycle are fairly complex to use and are fairly complex
to write, so it is good that we're going to have plenty of time to
clean and clarify all these things.</p>
<p>I started some performance analysis this week. Initially I started
working with <a href="https://anticdent.org/profiling-placement-in-docker.html">placement master in a
container</a>
but as I started making changes I moved back to
<a href="https://anticdent.org/profiling-wsgi-apps.html">container-less</a>.
What I discovered was that there is quite a bit of redundancy in the
code in the <code>objects</code> package that I was able to remove. For
example we were creating at least twice as many ProviderSummary
objects than required in a situation with multiple request groups.
It's likely there would have been more duplicates with more request
groups. That's improved in <a href="https://review.opendev.org/674254">this
change</a>, which is at the end of a
stack of several other like-minded improvements.</p>
<p>The improvements in that stack will not be obvious until the <a href="https://review.opendev.org/#/c/673513/">more
complex nested topology</a> is
generally available. My analysis was based on that topology.</p>
<p>Not to put too fine a point on it, but this kind of incremental
analysis and improvement is something I think we (the we that is the
community of OpenStack) should be doing far more often. It is
<em>incredibly</em> revealing about how the system works and opportunities
for making the code both work better and be easier to maintain.</p>
<p>One outcome of this work will be something like a <em>Deployment
Considerations</em> document to help people choose how to tweak their
placement deployment to match their needs. The simple answer is use
more web servers and more database servers, but that's often very
wasteful.</p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There is one <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And two <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
changes</a>.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New discoveries are added to the end. Merged stuff is removed.
Anything that has had no activity in 4 weeks has been removed.</p>
<ul>
<li>
<p><a href="https://review.openstack.org/#/q/topic:bug/1819923">https://review.openstack.org/#/q/topic:bug/1819923</a>
Nova: nova-manage: heal port allocations</p>
</li>
<li>
<p><a href="https://review.opendev.org/659233">https://review.opendev.org/659233</a>
Cyborg: Placement report</p>
</li>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/634551">https://review.opendev.org/634551</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/670112">https://review.opendev.org/670112</a>
Nova: WIP: Add a placement audit command</p>
</li>
<li>
<p><a href="https://review.opendev.org/586960">https://review.opendev.org/586960</a>
zun: [WIP] Use placement for unified resource management</p>
</li>
<li>
<p><a href="https://review.opendev.org/671478">https://review.opendev.org/671478</a>
Kayobe: Build placement images by default</p>
</li>
<li>
<p><a href="https://review.opendev.org/671312">https://review.opendev.org/671312</a>
blazar: Fix placement operations in multi-region deployments</p>
</li>
<li>
<p><a href="https://review.opendev.org/671793">https://review.opendev.org/671793</a>
Nova: libvirt: Start reporting PCPU inventory to placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/664689">https://review.opendev.org/664689</a>
Nova: get_ksa_adapter: nix by-service-type confgrp hack</p>
</li>
<li>
<p><a href="https://review.opendev.org/666202">https://review.opendev.org/666202</a>
Blazar: Create placement client for each request</p>
</li>
<li>
<p><a href="https://review.opendev.org/667952">https://review.opendev.org/667952</a>
nova: Support filtering of hosts by forbidden aggregates</p>
</li>
<li>
<p><a href="https://review.opendev.org/669079">https://review.opendev.org/669079</a>
blazar: Send global_request_id for tracing calls</p>
</li>
<li>
<p><a href="https://review.opendev.org/668252">https://review.opendev.org/668252</a>
Nova: Update HostState.*_allocation_ratio earlier</p>
</li>
<li>
<p><a href="https://review.opendev.org/670696">https://review.opendev.org/670696</a>
tempest: Add placement API methods for testing routed provider nets</p>
</li>
<li>
<p><a href="https://review.opendev.org/672678">https://review.opendev.org/672678</a>
openstack-helm: Build placement in OSH-images</p>
</li>
<li>
<p><a href="https://review.opendev.org/674129">https://review.opendev.org/674129</a>
Correct global_request_id sent to Placement</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>I started working with around approximately 20,000 providers this
week. Only 980,000 to go.</p>Profiling Placement in Docker2019-07-31T14:00:00+01:002019-07-31T14:00:00+01:00Chris Denttag:anticdent.org,2019-07-31:/profiling-placement-in-docker.html<p>Back in March, I wrote <a href="/profiling-wsgi-apps.html">Profiling WSGI
Apps</a>, describing one way to profile the
<a href="https://docs.openstack.org/placement/latest/">placement</a> service.
It was useful enough that a version of it was added to <a href="https://docs.openstack.org/placement/latest/contributor/testing.html#profiling">the
docs</a>.</p>
<p>Since then I've wanted something a bit more flexible. I maintain a
<a href="https://hub.docker.com/r/cdent/placedock">container for placement</a>
on Docker hub. When I …</p><p>Back in March, I wrote <a href="/profiling-wsgi-apps.html">Profiling WSGI
Apps</a>, describing one way to profile the
<a href="https://docs.openstack.org/placement/latest/">placement</a> service.
It was useful enough that a version of it was added to <a href="https://docs.openstack.org/placement/latest/contributor/testing.html#profiling">the
docs</a>.</p>
<p>Since then I've wanted something a bit more flexible. I maintain a
<a href="https://hub.docker.com/r/cdent/placedock">container for placement</a>
on Docker hub. When I want to profile recent master instead of code
in a proposed branch, using a container can be tidier. Since this
might be useful to others I thought I better write it down.</p>
<h1 id="get-and-confirm-the-image">Get and Confirm the Image</h1>
<p>Make sure you are on a host with docker and then get the latest
version of <code>placedock</code>:</p>
<div class="highlight"><pre><span></span>docker pull cdent/placedock
</pre></div>
<p>We can confirm the container is going to work with a quick run:</p>
<div class="highlight"><pre><span></span>docker run -it --env <span class="nv">OS_PLACEMENT_DATABASE__SYNC_ON_STARTUP</span><span class="o">=</span>True <span class="se">\</span>
--env <span class="nv">OS_API__AUTH_STRATEGY</span><span class="o">=</span>noauth2 <span class="se">\</span>
-p 127.0.0.1:8080:80 <span class="se">\</span>
--rm --name placement cdent/placedock
</pre></div>
<p>In another terminal check it is working:</p>
<div class="highlight"><pre><span></span>curl -s http://127.0.0.1:8080/ <span class="p">|</span>json_pp
</pre></div>
<p>should result in something similar to:</p>
<div class="highlight"><pre><span></span><span class="p">{</span>
<span class="nt">"versions"</span> <span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="nt">"min_version"</span> <span class="p">:</span> <span class="s2">"1.0"</span><span class="p">,</span>
<span class="nt">"links"</span> <span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="nt">"href"</span> <span class="p">:</span> <span class="s2">""</span><span class="p">,</span>
<span class="nt">"rel"</span> <span class="p">:</span> <span class="s2">"self"</span>
<span class="p">}</span>
<span class="p">],</span>
<span class="nt">"status"</span> <span class="p">:</span> <span class="s2">"CURRENT"</span><span class="p">,</span>
<span class="nt">"max_version"</span> <span class="p">:</span> <span class="s2">"1.36"</span><span class="p">,</span>
<span class="nt">"id"</span> <span class="p">:</span> <span class="s2">"v1.0"</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
</pre></div>
<p><code>ctrl-c</code> in the terminal that is running the container. We don't
want to use that one because we need one that will persist data
properly.</p>
<h1 id="dockerenv-for-convenience">Dockerenv for Convenience</h1>
<p>To enable persistence, a <code>dockerenv</code> file will be used to establish
configuration settings. The one I use, with comments:</p>
<div class="highlight"><pre><span></span><span class="cp"># Turn on debug logging</span>
<span class="n">OS_DEFAULT__DEBUG</span><span class="o">=</span><span class="n">True</span>
<span class="cp"># Don't use keystone for authentiation, instead pass</span>
<span class="cp"># 'x-auth-token: admin' headers</span>
<span class="n">OS_API__AUTH_STRATEGY</span><span class="o">=</span><span class="n">noauth2</span>
<span class="cp"># Make sure the database has the right tables on startup</span>
<span class="n">OS_PLACEMENT_DATABASE__SYNC_ON_STARTUP</span><span class="o">=</span><span class="n">True</span>
<span class="cp"># Connection to a remote database. The correct database URI depends on</span>
<span class="cp"># your environment.</span>
<span class="n">OS_PLACEMENT_DATABASE__CONNECTION</span><span class="o">=</span><span class="n">postgresql</span><span class="o">+</span><span class="nl">psycopg2</span><span class="p">:</span><span class="c1">//cdent@192.168.1.76/placement?client_encoding=utf8</span>
<span class="cp"># The directory where profile output should go. Leave this commented until</span>
<span class="cp"># sufficient data is present for a reasonable test.</span>
<span class="cp"># OS_WSGI_PROFILER=/profiler</span>
</pre></div>
<p>Create your own <code>dockerenv</code> file, set the database URI accordingly
(if you're not sure about what this might be, <a href="https://docs.openstack.org/placement/latest/contributor/quick-dev.html">Quick Placement
Development</a>
may be useful), and start the container back up. This time we will
use the <code>dockerenv</code> file and put the container in the background:</p>
<div class="highlight"><pre><span></span>docker run -idt -p 127.0.0.1:8080:80 <span class="se">\</span>
--env-file dockerenv <span class="se">\</span>
--rm --name placement <span class="se">\</span>
-v /tmp/profiler:/profiler <span class="se">\</span>
cdent/placedock
</pre></div>
<p>We've added a volume so that, eventually, profiler output can be
saved to the disk of your container host, rather than the container
itself. For the time being profiling is turned off because we don't
want to slow down the system while adding data to the system.</p>
<p>If you want to, confirm things are working again with:</p>
<div class="highlight"><pre><span></span>curl -s http://127.0.0.1:8080/ <span class="p">|</span>json_pp
</pre></div>
<h1 id="load-some-data">Load Some Data</h1>
<p><em>In some cases you don't need pre-existing data when profiling. If
that's the case, you can skip this step. What you need to use to set
up data may be very different from what I'm doing here.</em></p>
<p>Loading up the service with a bunch of data can be accomplished in
various ways. I use a combination of shell scripts and
<a href="https://gabbi.readthedocs.io">gabbi</a>. The shell script is
responsible for the dynamic data while gabbi is responsible the
static structure. Some <a href="https://review.opendev.org/673513">pending
changes</a> use the same system for
doing some performance testing for placement. We will borrow that
system. To speed things up a bit we'll use
<a href="https://www.gnu.org/software/parallel/">parallel</a>.</p>
<div class="highlight"><pre><span></span>seq <span class="m">1</span> <span class="m">100</span> <span class="p">|</span> <span class="se">\</span>
parallel <span class="s2">"./gate/perfload-nested-loader.sh http://127.0.0.1:8080 gate/gabbits/nested-perfload.yaml"</span>
</pre></div>
<blockquote>
<p><strong>Note:</strong> This will not work on a mac if you are using the built in
<code>uuidgen</code>. You may need a pipe to <code>tr [A-Z] [a-z]</code>.</p>
</blockquote>
<p>You can see how many providers you've created with a request like:</p>
<div class="highlight"><pre><span></span>curl -s -H <span class="s1">'x-auth-token: admin'</span> <span class="se">\</span>
-H <span class="s1">'openstack-api-version: placement latest'</span> <span class="se">\</span>
http://127.0.0.1:8080/resource_providers <span class="p">|</span> <span class="se">\</span>
json_pp<span class="p">|</span> grep -c <span class="s1">'"name"'</span>
</pre></div>
<p>Once you have a sufficient number of resource providers and anything
else you might like (such as allocations) in the system, you can
start profiling.</p>
<h1 id="profiling">Profiling</h1>
<p>When we were loading data we had profiling turned off. Now we'd like
to turn it on. Edit the <code>dockerenv</code> file to uncomment the
<code>OS_WSGI_PROFILER</code> line and then <code>docker kill placement</code> and run the
container again (using the same args as above). A <code>restart</code> will not
work because we've change the environment.</p>
<p>Make a query, such as:</p>
<div class="highlight"><pre><span></span>curl http://127.0.0.1:8080/
</pre></div>
<p>and look in <code>/tmp/profiler</code> for the profile output, the filename
should looking something like this:</p>
<div class="highlight"><pre><span></span>GET.root.19ms.1564579909.prof
</pre></div>
<p>If you have <a href="https://pypi.org/project/snakeviz/">snakeviz</a> installed
you can inspect the profiling info from a browser (as described in
the <a href="/profiling-wsgi-apps.html">previous blog post</a>:</p>
<div class="highlight"><pre><span></span>snakeviz GET.root.19ms.1564579909.prof
</pre></div>
<p>That's not a very interesting request. It doesn't exercise much of
the code nor access the database. If the system has been loaded with
data as above, the following will query it:</p>
<div class="highlight"><pre><span></span>curl -H <span class="s1">'x-auth-token: admin'</span> <span class="se">\</span>
-H <span class="s1">'openstack-api-version: placement latest'</span> <span class="se">\</span>
<span class="s2">"http://127.0.0.1:8080/allocation_candidates?\</span>
<span class="s2">resources=DISK_GB:10&\</span>
<span class="s2">required=COMPUTE_VOLUME_MULTI_ATTACH&\</span>
<span class="s2">resources_COMPUTE=VCPU:1,MEMORY_MB:256&\</span>
<span class="s2">required_COMPUTE=CUSTOM_FOO&\</span>
<span class="s2">resources_FPGA=FPGA:1&\</span>
<span class="s2">group_policy=none&\</span>
<span class="s2">same_subtree=_COMPUTE,_FPGA"</span>
</pre></div>
<p>and then:</p>
<div class="highlight"><pre><span></span>snakeviz GET.allocation_candidates.792ms.1564581384.prof
</pre></div>
<p><img alt="snakeviz sunburst" src="/images/snakeviz.png"></p>Placement Update 19-292019-07-26T16:40:00+01:002019-07-26T16:40:00+01:00Chris Denttag:anticdent.org,2019-07-26:/placement-update-19-29.html<p>Welcome to a rushed pupdate 19-29. My morning was consumed by other
things.</p>
<p>A reminder: The Placement project holds office hours every Wednesday
at 1500 UTC in the <code>#openstack-placement</code> IRC channel. If you have a
topic that needs some synchronous discussion, then is an ideal time.
Just start talking!</p>
<h1 id="most-important">Most …</h1><p>Welcome to a rushed pupdate 19-29. My morning was consumed by other
things.</p>
<p>A reminder: The Placement project holds office hours every Wednesday
at 1500 UTC in the <code>#openstack-placement</code> IRC channel. If you have a
topic that needs some synchronous discussion, then is an ideal time.
Just start talking!</p>
<h1 id="most-important">Most Important</h1>
<p>The two main things on the Placement radar are implementing Consumer
Types and cleanups, performance analysis, and documentation related
to nested resource providers.</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p>The <a href="https://docs.openstack.org/api-ref/placement/">api-ref</a> has
moved to <code>docs.openstack.org</code> from <code>developer.openstack.org</code>.
Redirects are in place.</p>
</li>
<li>
<p>Both traits and resource classes are <a href="https://review.opendev.org/672298">now
cached</a> per request, allowing
for some name to id and id to name optimizations.</p>
</li>
<li>
<p>A new zuul template is <a href="https://review.opendev.org/671257">being
used</a> in placement that means
fewer irrelevant tempest tests are run on placement changes.</p>
</li>
</ul>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 21 (-1) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
2 (0) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 5 (0)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 10
(0) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
4 (0) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 17 (0).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 4 (0).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<p>osc-placement is currently behind by 12 microversions.</p>
<ul>
<li><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of. There's been some useful
discussion about how to achieve this, and a consensus has emerged
on how to get the best results.</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a>
A WIP, as microversion 1.37, has started.</li>
</ul>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we are
making this cycle are fairly complex to use and are fairly complex
to write, so it is good that we're going to have plenty of time to
clean and clarify all these things.</p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There are two <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And zero <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
changes</a>
(yay!).</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New discoveries are added to the end. Merged stuff is removed.
Anything that has had no activity in 4 weeks has been removed.</p>
<ul>
<li>
<p><a href="https://review.openstack.org/#/q/topic:bug/1819923">https://review.openstack.org/#/q/topic:bug/1819923</a>
Nova: nova-manage: heal port allocations</p>
</li>
<li>
<p><a href="https://review.opendev.org/659233">https://review.opendev.org/659233</a>
Cyborg: Placement report</p>
</li>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/656023">https://review.opendev.org/656023</a>
Nova: Use OpenStack SDK for placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/634551">https://review.opendev.org/634551</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/664689">https://review.opendev.org/664689</a>
Nova: get_ksa_adapter: nix by-service-type confgrp hack</p>
</li>
<li>
<p><a href="https://review.opendev.org/666202">https://review.opendev.org/666202</a>
Blazar: Create placement client for each request</p>
</li>
<li>
<p><a href="https://review.opendev.org/667952">https://review.opendev.org/667952</a>
nova: Support filtering of hosts by forbidden aggregates</p>
</li>
<li>
<p><a href="https://review.opendev.org/669079">https://review.opendev.org/669079</a>
blazar: Send global_request_id for tracing calls</p>
</li>
<li>
<p><a href="https://review.opendev.org/668252">https://review.opendev.org/668252</a>
Nova: Update HostState.*_allocation_ratio earlier</p>
</li>
<li>
<p><a href="https://review.opendev.org/670112">https://review.opendev.org/670112</a>
Nova: WIP: Add a placement audit command</p>
</li>
<li>
<p><a href="https://review.opendev.org/586960">https://review.opendev.org/586960</a>
zun: [WIP] Use placement for unified resource management</p>
</li>
<li>
<p><a href="https://review.opendev.org/671478">https://review.opendev.org/671478</a>
Kayobe: Build placement images by default</p>
</li>
<li>
<p><a href="https://review.opendev.org/671312">https://review.opendev.org/671312</a>
blazar: Fix placement operations in multi-region deployments</p>
</li>
<li>
<p><a href="https://review.opendev.org/672870">https://review.opendev.org/672870</a>
Watcher: Getting data from placement when updating datamodel</p>
</li>
<li>
<p><a href="https://review.opendev.org/671793">https://review.opendev.org/671793</a>
Nova: libvirt: Start reporting PCPU inventory to placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/672649">https://review.opendev.org/672649</a>
chef: placement_api: create valid apache config before installing package</p>
</li>
<li>
<p><a href="https://review.opendev.org/670696">https://review.opendev.org/670696</a>
tempest: Add placement API methods for testing routed provider nets</p>
</li>
<li>
<p><a href="https://review.opendev.org/672678">https://review.opendev.org/672678</a>
openstack-helm: Build placement in OSH-images</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>If we get the chance, it will be interesting to start working with
placement with 1 million providers. Just to see.</p>Placement Update 19-282019-07-19T12:13:00+01:002019-07-19T12:13:00+01:00Chris Denttag:anticdent.org,2019-07-19:/placement-update-19-28.html<p>This is pupdate 19-28. Next week is the Train-2 milestone.</p>
<h1 id="most-important">Most Important</h1>
<p>Based on the discussion on the <a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007654.html">PTG attendance
thread</a>
and the notes on the <a href="https://etherpad.openstack.org/p/placement-shanghai-ptg">related
etherpad</a>
I'm going to tell the Foundation there will be approximately seven
Placement team members at Shanghai but formal space or scheduling
will …</p><p>This is pupdate 19-28. Next week is the Train-2 milestone.</p>
<h1 id="most-important">Most Important</h1>
<p>Based on the discussion on the <a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007654.html">PTG attendance
thread</a>
and the notes on the <a href="https://etherpad.openstack.org/p/placement-shanghai-ptg">related
etherpad</a>
I'm going to tell the Foundation there will be approximately seven
Placement team members at Shanghai but formal space or scheduling
will not be required. Instead any necessary discussions will be
arranged on premise. If you disagree with this, please speak up
soon.</p>
<p>The main pending feature is consumer types, see below.</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p>A bug in the resource class cache used in the placement server was
found and <a href="https://review.opendev.org/671341">fixed</a>. It will be
interesting to see how this impacts performance. While it
increases database reads by one (for most requests) it removes a
process-wide lock, so things could improve in threaded servers.</p>
</li>
<li>
<p>os-resource-classes
<a href="https://pypi.org/project/os-resource-classes/">0.5.0</a> was
released, adding FPGA and PGPU resource classes.</p>
<p>It's been discussed in IRC that we may wish to make 1.x releases
of both os-resource-classes and os-traits at some point to make
it clear that they are "real". If we do this, I recommend we do
it near a cycle boundary.</p>
</li>
<li>
<p>An <a href="https://review.opendev.org/669309">integrated-gate-placement</a>
zuul template has merged. A <a href="https://review.opendev.org/671257">placement change to use
it</a> is ready and waiting to
merge. This avoids running some tests which are unrelated; for
example, cinder-only tests.</p>
</li>
</ul>
<h1 id="specsfeatures">Specs/Features</h1>
<p>Since spec freeze for most projects is next week and placement has
merged all its specs, until U opens, I'm going to skip this section.</p>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 22 (-1) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
2 (-1) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 5 (0)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 10
(-1) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
4 (0) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 17 (1).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 4 (0).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<p>osc-placement is currently behind by 11 microversions.</p>
<ul>
<li><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of. There's been some useful
discussion about how to achieve this, and a consensus has emerged
on how to get the best results.</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a>
This is currently the migration to add the necessary table and
column.</li>
</ul>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we are
making this cycle are fairly complex to use and are fairly complex
to write, so it is good that we're going to have plenty of time to
clean and clarify all these things.</p>
<p>As mentioned for a few weeks, one of the important cleanup tasks
that is not yet in progress is updating the
<a href="https://opendev.org/openstack/placement/src/branch/master/gate/gabbits/nested-perfload.yaml">gabbit</a>
that creates the nested topology that's used in nested performance
testing. We've <a href="http://lists.starlingx.io/pipermail/starlingx-discuss/2019-July/005330.html">asked the startlingx
community</a>
for input.</p>
<p>Another cleanup that needs to start is satisfying the community wide
goal of <a href="https://storyboard.openstack.org/#!/story/2006110">PDF doc
generation</a>. I
did some experimenting on this and while I was able to get a book
created, the number of errors, warnings, and manual interventions
required meant I gave up until there's time to do a more in-depth
exploration and learn the tools.</p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There are two <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And zero <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
changes</a>
(yay!).</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New discoveries are added to the end. Merged stuff is removed.
Anything that has had no activity in 4 weeks has been removed.</p>
<ul>
<li>
<p><a href="https://review.openstack.org/#/q/topic:bug/1819923">https://review.openstack.org/#/q/topic:bug/1819923</a>
Nova: nova-manage: heal port allocations</p>
</li>
<li>
<p><a href="https://review.openstack.org/650188">https://review.openstack.org/650188</a>
nova-spec: Allow compute nodes to use DISK_GB from shared storage RP</p>
</li>
<li>
<p><a href="https://review.opendev.org/659233">https://review.opendev.org/659233</a>
Cyborg: Placement report</p>
</li>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/656023">https://review.opendev.org/656023</a>
Nova: Use OpenStack SDK for placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/612497">https://review.opendev.org/612497</a>
Nova: Spec: Provider config YAML file</p>
</li>
<li>
<p><a href="https://review.opendev.org/634551">https://review.opendev.org/634551</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/664689">https://review.opendev.org/664689</a>
Nova: get_ksa_adapter: nix by-service-type confgrp hack</p>
</li>
<li>
<p><a href="https://review.opendev.org/666202">https://review.opendev.org/666202</a>
Blazar: Create placement client for each request</p>
</li>
<li>
<p><a href="https://review.opendev.org/667952">https://review.opendev.org/667952</a>
nova: Support filtering of hosts by forbidden aggregates</p>
</li>
<li>
<p><a href="https://review.opendev.org/669079">https://review.opendev.org/669079</a>
blazar: Send global_request_id for tracing calls</p>
</li>
<li>
<p><a href="https://review.opendev.org/668252">https://review.opendev.org/668252</a>
Nova: Update HostState.*_allocation_ratio earlier</p>
</li>
<li>
<p><a href="https://review.opendev.org/670112">https://review.opendev.org/670112</a>
Nova: WIP: Add a placement audit command</p>
</li>
<li>
<p><a href="https://review.opendev.org/586960">https://review.opendev.org/586960</a>
zun: [WIP] Use placement for unified resource management</p>
</li>
<li>
<p><a href="https://review.opendev.org/671371">https://review.opendev.org/671371</a>
neutron: Filter placement API endpoint by type too</p>
</li>
<li>
<p><a href="https://review.opendev.org/671638">https://review.opendev.org/671638</a>
OSA: Install placement osc plugin on utility machines</p>
</li>
<li>
<p><a href="https://review.opendev.org/671478">https://review.opendev.org/671478</a>
Kayobe: Build placement images by default</p>
</li>
<li>
<p><a href="https://review.opendev.org/671312">https://review.opendev.org/671312</a>
blazar: Fix placement operations in multi-region deployments</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>If you've done any performance or scale testing with placement, I'd
love to hear about your observations. Please let me know.</p>Eight Hour Day Update2019-07-12T13:00:00+01:002019-07-12T13:00:00+01:00Chris Denttag:anticdent.org,2019-07-12:/eight-hour-day-update.html<p>Since <a href="/openstack-denver-summit-reflection.html">Denver in early
May</a> I've been running a
timer to limit my work day to eight hours. I've stuck to it pretty
well, long enough that I have a few observations.</p>
<p>Some of the expected positive outcomes are there: I have more time
to attend to non-work tasks like …</p><p>Since <a href="/openstack-denver-summit-reflection.html">Denver in early
May</a> I've been running a
timer to limit my work day to eight hours. I've stuck to it pretty
well, long enough that I have a few observations.</p>
<p>Some of the expected positive outcomes are there: I have more time
to attend to non-work tasks like feeding myself, getting a bit more
exercise, and making plans and actions for things around my home.</p>
<p>But there are some negatives which suggest further effort is
required.</p>
<p>The one that is on my mind today is that with only eight hours of
continuous work in a day, it's been difficult to get anything of
substance done. Not that I'm getting nothing done. Rather, the time
I have available is mostly consumed by reputedly urgent
requests—that are initially small but turn out not to be—from co-workers
both internal to $EMPLOYER and in the OpenStack community.</p>
<p>In the past I would attend to these requests, clear them off the
plate, and then do what I felt to be "the real work" (which could be
defined, vaguely, as "improving OpenStack for the long term").</p>
<p>Now that I have a time limit, I rarely get to the point where I have
a clean plate. If I do there's not enough time to gain the focus
and flow required to do "the real work". As a result, my day to day
satisfaction is poor.</p>
<p>I can think of a few strategies for resolving this, but both will be
difficult to integrate with social mores in my daily environments:</p>
<ul>
<li>
<p>Reserve entire days without attention to IRC, Slack and perhaps
even email. That is, avoid interruption by being unreachable. This
will be hard to do. The majority of the population in these
environments is addicted to 24 hour synchronous communication and
expects the same from others. People get used to it in some kind
of bizarre form of Stockholm Syndrome and synchronous becomes the
only reliable way to reach them.</p>
<p>24 hours, seven days a week contact is not how work is supposed
to work. If you are a member of a team and you work this way,
you are effectively encouraging, and in some cases requiring,
other members of your team to work the same way.</p>
</li>
<li>
<p>Enforce a queuing mechanism. Don't let other people turn me into a
stack on which they can put themselves.</p>
</li>
</ul>
<p>These are both hard because there is always a perceived urgency.
Sometimes real, sometimes not. Denying or resisting that is easily
perceived as rude or unhelpful.</p>
<p>One of the reasons I write the <a href="/placement-update-19-27.html">placement
updates</a> is to make it clear there is
a queue of placement-related work for people who either cannot or do
not want to be a part of the synchronous flow of information.</p>
<p>I need more tools like that.</p>
<p>I'm not going to go back to greater than eight hour days. Until
I find some better ways to manage tasks and inputs my apologies (to
me and to you) for not getting the good stuff done.</p>Placement Update 19-272019-07-12T10:56:00+01:002019-07-12T10:56:00+01:00Chris Denttag:anticdent.org,2019-07-12:/placement-update-19-27.html<p>Pupdate 19-27 is here and now.</p>
<h1 id="most-important">Most Important</h1>
<p>Of the features we planned to do this cycle, all are done save one:
consumer types (in progress, see below). This means we have a good
opportunity to focus on documentation, performance, and improving
the codebase for maintainability. You do not need …</p><p>Pupdate 19-27 is here and now.</p>
<h1 id="most-important">Most Important</h1>
<p>Of the features we planned to do this cycle, all are done save one:
consumer types (in progress, see below). This means we have a good
opportunity to focus on documentation, performance, and improving
the codebase for maintainability. You do not need permission to work
on these things. If you find a problem and know how to fix it, fix
it. If you are not sure about the solution, please discuss it on
this email list or in the <code>#openstack-placement</code> IRC channel.</p>
<p>This also means we're in a good position to help review changes that
use placement in other projects.</p>
<p>The Foundation needs to know how much, if any, Placement time will
be needed in Shanghai. I started a
<a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007654.html">thread</a>
and an
<a href="https://etherpad.openstack.org/p/placement-shanghai-ptg">etherpad</a>.</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p>The <code>same_subtree</code> query parameter has merged as <a href="https://docs.openstack.org/placement/latest/placement-api-microversion-history.html#support-same-subtree-queryparam-on-get-allocation-candidates">microversion
1.36</a>.
This enables a form of nested provider affinity: "these providers
must all share the same ancestor".</p>
</li>
<li>
<p>The placement projects have been updated (pending merge) to match
the <a href="https://governance.openstack.org/tc/goals/train/python3-updates.html">Python 3 test runtimes for
Train</a>
community wide goal. Since the projects have been Python3-enabled
from the start, this was mostly a matter of aligning configurations
with community-wide norms. When Python 3.8 becomes available we
should get on that sooner than later to catch issues early.</p>
</li>
</ul>
<h1 id="specsfeatures">Specs/Features</h1>
<p>All placement specs have merged. Thanks to everyone for the frequent
reviews and quick followups.</p>
<p>Some non-placement specs are listed in the Other section below.</p>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 23 (0) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
3 (1) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 5 (0)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 11
(0) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
4 (0) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 16 (0).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 4 (0).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<p>osc-placement is currently behind by 11 microversions.</p>
<ul>
<li><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of. This is stuck and needs input
from users of the tool.</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a>
This is currently the migration to add the necessary table and
column.</li>
</ul>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we are
making this cycle are fairly complex to use and are fairly complex
to write, so it is good that we're going to have plenty of time to
clean and clarify all these things.</p>
<p>As mentioned last week, one of the important cleanup tasks that is not
yet in progress is updating the
<a href="https://opendev.org/openstack/placement/src/branch/master/gate/gabbits/nested-perfload.yaml">gabbit</a>
that creates the nested topology that's used in nested performance
testing. The topology there is simple, unrealistic, and doesn't
sufficiently exercise the several features that may be used during a
query that desires a nested response. This needs to be someone who
is more closely related to real world use of nested than me. efried?
gibi?</p>
<p>Another cleanup that needs to start is satisfying the community wide
goal of <a href="https://storyboard.openstack.org/#!/story/2006110">PDF doc
generation</a>.
Anyone know if there is a cookbook for this? </p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There are three <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And one <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
change</a>.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New discoveries are added to the end. Merged stuff is removed.
Anything that has had no activity in 4 weeks has been removed.</p>
<ul>
<li>
<p><a href="https://review.openstack.org/#/q/topic:bug/1819923">https://review.openstack.org/#/q/topic:bug/1819923</a>
Nova: nova-manage: heal port allocations</p>
</li>
<li>
<p><a href="https://review.openstack.org/650188">https://review.openstack.org/650188</a>
nova-spec: Allow compute nodes to use DISK_GB from shared storage RP</p>
</li>
<li>
<p><a href="https://review.opendev.org/659233">https://review.opendev.org/659233</a>
Cyborg: Placement report</p>
</li>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/656023">https://review.opendev.org/656023</a>
Nova: Use OpenStack SDK for placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/612497">https://review.opendev.org/612497</a>
Nova: Spec: Provider config YAML file</p>
</li>
<li>
<p><a href="https://review.opendev.org/634551">https://review.opendev.org/634551</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/664689">https://review.opendev.org/664689</a>
Nova: get_ksa_adapter: nix by-service-type confgrp hack</p>
</li>
<li>
<p><a href="https://review.opendev.org/664862">https://review.opendev.org/664862</a>
OSA: Add nova placement to placement migration</p>
</li>
<li>
<p><a href="https://review.opendev.org/657796">https://review.opendev.org/657796</a>
Nova: Defaults missing group_policy to 'none'</p>
</li>
<li>
<p><a href="https://review.opendev.org/666202">https://review.opendev.org/666202</a>
Blazar: Create placement client for each request</p>
</li>
<li>
<p><a href="https://review.opendev.org/669309">https://review.opendev.org/669309</a>
tempest: Define the Integrated-gate-placement gate template</p>
</li>
<li>
<p><a href="https://review.opendev.org/668263">https://review.opendev.org/668263</a>
Nova: Restore RT.old_resources if ComputeNode.save() fails</p>
</li>
<li>
<p><a href="https://review.opendev.org/667952">https://review.opendev.org/667952</a>
nova: Support filtering of hosts by forbidden aggregates</p>
</li>
<li>
<p><a href="https://review.opendev.org/669079">https://review.opendev.org/669079</a>
blazar: Send global_request_id for tracing calls</p>
</li>
<li>
<p><a href="https://review.opendev.org/668252">https://review.opendev.org/668252</a>
Nova: Update HostState.*_allocation_ratio earlier</p>
</li>
<li>
<p><a href="https://review.opendev.org/670105">https://review.opendev.org/670105</a>
neutron: segments: fix rp inventory update</p>
</li>
<li>
<p><a href="https://review.opendev.org/670112">https://review.opendev.org/670112</a>
Nova: WIP: Add a placement audit command</p>
</li>
<li>
<p><a href="https://review.opendev.org/668526">https://review.opendev.org/668526</a>
OSA: Add placement client for neutron</p>
</li>
<li>
<p><a href="https://review.opendev.org/586960">https://review.opendev.org/586960</a>
zun: [WIP] Use placement for unified resource management</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>A colleague suggested yesterday that the universe doesn't have an
over subscription problem, rather there's localized contention, and
what we really have is a placement problem.</p>Placement Update 19-262019-07-05T14:00:00+01:002019-07-05T14:00:00+01:00Chris Denttag:anticdent.org,2019-07-05:/placement-update-19-26.html<p>Pupdate 19-26. Next week is R-15, two weeks until Train milestone 2.</p>
<h1 id="most-important">Most Important</h1>
<p>The <a href="https://docs.openstack.org/placement/latest/specs/train/approved/2005575-nested-magic-1.html">spec for nested
magic</a>
merged and significant progress has been made in the implementation.
That work is nearly ready to merge (see below), after a few more
reviews. Once that happens one of our most …</p><p>Pupdate 19-26. Next week is R-15, two weeks until Train milestone 2.</p>
<h1 id="most-important">Most Important</h1>
<p>The <a href="https://docs.openstack.org/placement/latest/specs/train/approved/2005575-nested-magic-1.html">spec for nested
magic</a>
merged and significant progress has been made in the implementation.
That work is nearly ready to merge (see below), after a few more
reviews. Once that happens one of our most important tasks will be
experimenting with that code to make sure it fully addresses the
uses cases, has proper documentation (including "how do I use
this?"), and is properly evaluated for performance and
maintainability.</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p>The implementation for mappings in allocation candidates had a bug
which Eric <a href="https://review.opendev.org/668302">found</a> and fixed
and then I realized there was a <a href="https://review.opendev.org/668724">tidier way to do
it</a>. This then led to the
<code>same_subtree</code> work needing to manage less information, because it
was already there.</p>
</li>
<li>
<p>The spec for <a href="https://docs.openstack.org/placement/latest/specs/train/approved/2005473-support-consumer-types.html">Consumer
Types</a>
merged and work <a href="https://review.opendev.org/669170">has
started</a>.</p>
</li>
<li>
<p>We're using os-traits 0.15.0 now.</p>
</li>
<li>
<p>There's a <a href="https://review.opendev.org/665695">framework in place</a>
for nested resource provider peformance testing. We need to update
the provider topology to reflect real world situations (more on
that below).</p>
</li>
<li>
<p>The <code>root_required</code> query parameter on <code>GET
/allocation_candidates</code> has been merged as <a href="https://docs.openstack.org/placement/latest/placement-api-microversion-history.html#support-root-required-queryparam-on-get-allocation-candidates">microversion
1.35</a>.</p>
</li>
<li>
<p>I've sent <a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007527.html">an
email</a>
announcing my intent to not go to the Shangai (or any other)
summit, and what changes that could imply for how Placement does
the PTG.</p>
</li>
</ul>
<h1 id="specsfeatures">Specs/Features</h1>
<p>All placement specs have merged. Thanks to everyone for the frequent
reviews and quick followups. We've been maintaining some good
velocity.</p>
<p>Some non-placement specs are listed in the Other section below.</p>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 23 (3) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
2 (-2) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 5 (0)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 11
(0) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
4 (1) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 16 (0).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 4 (-1).</p>
</li>
</ul>
<h1 id="osc-placement">osc-placement</h1>
<p>osc-placement is currently behind by 11 microversions.</p>
<ul>
<li><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of.</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="nested-magic">Nested Magic</h2>
<p>These are the features required by modern nested resource provider
use cases. We've merged mappings in allocation candidates and
<code>root_required</code>. <code>same_subtree</code> and resourceless request groups are
what's left and they are in:</p>
<ul>
<li><a href="https://review.opendev.org/668376">https://review.opendev.org/668376</a>
Support <code>same_subtree</code> queryparam</li>
</ul>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting.</p>
<ul>
<li><a href="https://review.opendev.org/#/q/topic:bp/support-consumer-types">https://review.opendev.org/#/q/topic:bp/support-consumer-types</a></li>
</ul>
<h2 id="cleanup">Cleanup</h2>
<p>Cleanup is an overarching theme related to improving documentation,
performance and the maintainability of the code. The changes we are
making this cycle are fairly complex to use and are fairly complex
to write, so it is good that we're going to have plenty of time to
clean and clarify all these things.</p>
<p>As mentioned above, one of the important cleanup tasks that is not
yet in progress is updating the
<a href="https://opendev.org/openstack/placement/src/branch/master/gate/gabbits/nested-perfload.yaml">gabbit</a>
that creates the nested topology that's used in nested performance
testing. The topology there is simple, unrealistic, and doesn't
sufficiently exercise the several features that may be used during a
query that desires a nested response.</p>
<p>Recently I've been seeing that the <code>placement-perfload</code> job is
giving results that vary between <code>N</code> and <code>N*2</code> (usually .5 and 1
seconds) and the difference that I can discern is the type of CPUs
being presented by the host (same number of CPUs (8) but different
type). This supports something we've been theorizing for a while:
when dealing with large result sets we are CPU bound processing the
several large result sets returned by the database. Further
profiling required…</p>
<p>Another cleanup that needs to start is satisfying the community wide
goal of <a href="https://storyboard.openstack.org/#!/story/2006110">PDF doc
generation</a>.</p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There are three <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And one <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
change</a>.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New discoveries are added to the end. Merged stuff is removed.
Anything that has had no activity in 4 weeks has been removed.</p>
<ul>
<li>
<p><a href="https://review.openstack.org/#/q/topic:bug/1819923">https://review.openstack.org/#/q/topic:bug/1819923</a>
Nova: nova-manage: heal port allocations</p>
</li>
<li>
<p><a href="https://review.openstack.org/650188">https://review.openstack.org/650188</a>
nova-spec: Allow compute nodes to use DISK_GB from shared storage RP</p>
</li>
<li>
<p><a href="https://review.opendev.org/659233">https://review.opendev.org/659233</a>
Cyborg: Placement report</p>
</li>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: WIP: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/656023">https://review.opendev.org/656023</a>
Nova: Use OpenStack SDK for placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/612497">https://review.opendev.org/612497</a>
Nova: Spec: Provider config YAML file</p>
</li>
<li>
<p><a href="https://review.opendev.org/634551">https://review.opendev.org/634551</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/664689">https://review.opendev.org/664689</a>
Nova: get_ksa_adapter: nix by-service-type confgrp hack</p>
</li>
<li>
<p><a href="https://review.opendev.org/664862">https://review.opendev.org/664862</a>
OSA: Add nova placement to placement migration</p>
</li>
<li>
<p><a href="https://review.opendev.org/657796">https://review.opendev.org/657796</a>
Nova: Defaults missing group_policy to 'none'</p>
</li>
<li>
<p><a href="https://review.opendev.org/666202">https://review.opendev.org/666202</a>
Blazar: Create placement client for each request</p>
</li>
<li>
<p><a href="https://review.opendev.org/668930">https://review.opendev.org/668930</a>
tempest: Define the Integrated-gate-networking gate template</p>
</li>
<li>
<p><a href="https://review.opendev.org/669309">https://review.opendev.org/669309</a>
tempest: Define the Integrated-gate-placement gate template</p>
</li>
<li>
<p><a href="https://review.opendev.org/668263">https://review.opendev.org/668263</a>
Nova: Restore RT.old_resources if ComputeNode.save() fails</p>
</li>
<li>
<p><a href="https://review.opendev.org/669188">https://review.opendev.org/669188</a>
Remove assumption of http error if consumer not exists</p>
</li>
<li>
<p><a href="https://review.opendev.org/669253">https://review.opendev.org/669253</a>
TripleO: Add new parameter NovaSchedulerLimitTenantsToPlacementAggregate</p>
</li>
<li>
<p><a href="https://review.opendev.org/669252">https://review.opendev.org/669252</a>
puppet-nova: Expose limit_tenants_to_placement_aggregate parameter</p>
</li>
<li>
<p><a href="https://review.opendev.org/667952">https://review.opendev.org/667952</a>
nova: Support filtering of hosts by forbidden aggregates</p>
</li>
<li>
<p><a href="https://review.opendev.org/669079">https://review.opendev.org/669079</a>
blazar: Send global_request_id for tracing calls</p>
</li>
<li>
<p><a href="https://review.opendev.org/667417">https://review.opendev.org/667417</a>
nova: Implement update_provider_tree for hyperv</p>
</li>
<li>
<p><a href="https://review.opendev.org/668639">https://review.opendev.org/668639</a>
watcher: Improve Compute Data Model</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/pre-filter-disabled-computes">https://review.opendev.org/#/q/topic:bp/pre-filter-disabled-computes</a>
Nova: pre filter disable computes</p>
</li>
<li>
<p><a href="https://review.opendev.org/668252">https://review.opendev.org/668252</a>
Nova: Update HostState.*_allocation_ratio earlier</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>This space left intentionally blank.</p>Remote Maintainership2019-07-03T15:15:00+01:002019-07-03T15:15:00+01:00Chris Denttag:anticdent.org,2019-07-03:/remote-maintainership.html<p><em>This is a followup to <a href="/more-on-maintainership.html">More on
Maintainership</a> and <a href="/openstack-denver-summit-reflection.html">OpenStack Denver
Summit Reflection</a>.
Together these are starting to form the foundation of a series on
open source collaboration in the face of the climate emergency and
political and social inclusiveness. Starting...</em></p>
<p>I've decided, if possible, to stop flying to technology …</p><p><em>This is a followup to <a href="/more-on-maintainership.html">More on
Maintainership</a> and <a href="/openstack-denver-summit-reflection.html">OpenStack Denver
Summit Reflection</a>.
Together these are starting to form the foundation of a series on
open source collaboration in the face of the climate emergency and
political and social inclusiveness. Starting...</em></p>
<p>I've decided, if possible, to stop flying to technology or other
professional conferences. If I can get there by train or boat,
perhaps, but all the evidence I can gather suggests that the
environmental cost of conference attendance negates the value,
especially when:</p>
<ul>
<li>There are plenty of other costs, as discussed in my <a href="/openstack-denver-summit-reflection.html">Denver
report</a> and further,
below.</li>
<li>We have the technologies to collaborate remotely with greater
benefits and productivity than having a self-congratulatory party
with a few hundred or thousand of our not-so-closest friends, most
of whom burn a bunch of energy to get there and be there.</li>
</ul>
<p>There are—of course—advantages to being in person, especially in
terms of relationship building, but what's the point of building a
lasting friendship if you die soon after in a cataclysmic weather
event?</p>
<p>I've been thinking about this for a long time, but a recent article
in The Guardian, <a href="https://www.theguardian.com/science/2019/jun/29/no-flights-four-day-week-climate-scientists-home-save-planet">No flights, a four-day week and living off-grid:
what climate scientists do at home to save the
plane</a> crystallized the issues and
options for me. The article includes a section from Dave Reay, the
author of <a href="http://www.ghgonline.org/flyingaea.pdf">New Directions: Flying in the face of the climate change
convention</a>. He hasn't flown
since 2004.</p>
<p>Further, I can't ignore the social and political implications of
conference attendance. Wherever they are, conferences create a
separate space of privileged attendees and sometimes (often?) they
are held in places that I don't want to give economic or other
support. China's recent actions in Hong Kong and
<a href="https://www.theguardian.com/world/2019/jul/02/chinese-border-guards-surveillance-app-tourists-phones">Xinjiang</a> are intolerable and the onerous
list of actions colleagues have advised on how to safely use the
internet (and thus "do work") there do not inspire a
willingness to attend the next <a href="https://www.openstack.org/summit/shanghai-2019">OpenStack summit in
Shanghai</a>. Not that
many (any?) other countries are much better these days.</p>
<p>I'll need to consider if being unwilling to travel to summits takes
me out of the running for being a PTL or otherwise relevant in the
OpenStack community. I certainly hope not, both for my sake and the
sake of the community. We need as many, and as many different,
people as we can get and there are others like me.</p>
<p>I've been doing remote collaboration since the start of this
century, I even co-founded a <a href="https://tank.peermore.com/tanks/Collaboration/Blue%20Oxen%20Associates">think
tank</a>
focused on high-performance and frequently-remote collaboration. I
think I can help make it go.</p>Placement Update 19-242019-06-21T12:20:00+01:002019-06-21T12:20:00+01:00Chris Denttag:anticdent.org,2019-06-21:/placement-update-19-24.html<p>Here's a 19-24 pupdate. Last week I said there wouldn't be one this
week. I was wrong. There won't be one next week. I'm taking the week
off to reset (I hope). I've tried to make sure that there's nothing
floating about in Placement land that is blocking on me …</p><p>Here's a 19-24 pupdate. Last week I said there wouldn't be one this
week. I was wrong. There won't be one next week. I'm taking the week
off to reset (I hope). I've tried to make sure that there's nothing
floating about in Placement land that is blocking on me.</p>
<h1 id="most-important">Most Important</h1>
<p>The <a href="https://review.opendev.org/662191">spec for nested
magic</a> is close. What it needs
now is review from people outside the regulars to make sure it
doesn't suffer from tunnel vision. The features discussed form the
foundation for Placement being able to service queries for real
world uses of nested providers. Placement can already model nested
providers but asking for the right one has needed some work.</p>
<h1 id="editorial">Editorial</h1>
<p>I read an article this morning which touches on the importance of
considering <a href="https://techbeacon.com/app-dev-testing/forget-monoliths-vs-microservices-cognitive-load-what-matters">cognitive load in software
design</a>.
It's fully of glittering generalities, but it reminds me of some of
the reasons why it was important to keep a solid boundary between
Nova and Placement and why, now that Placement is extracted, the
complexity of this nested magic is something we need to measure
against the cognitive load it will induce in the people who come
after us as developers and users.</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p>Support for mappings in allocation candidates has merged as
<a href="https://docs.openstack.org/placement/latest/placement-api-microversion-history.html#request-group-mappings-in-allocation-candidates">microversion
1.34</a>.</p>
</li>
<li>
<p>Gibi made it so <a href="https://docs.openstack.org/osprofiler/latest/">OSProfiler</a>
works with placement again.</p>
</li>
</ul>
<h1 id="specsfeatures">Specs/Features</h1>
<ul>
<li>
<p><a href="https://review.opendev.org/654799">https://review.opendev.org/654799</a>
Support Consumer Types. This has some open questions that need to
be addressed, but we're still go on the general idea.</p>
</li>
<li>
<p><a href="https://review.opendev.org/662191">https://review.opendev.org/662191</a>
Spec for nested magic 1. See "Most Important" above.</p>
</li>
</ul>
<p>Some non-placement specs are listed in the Other section below.</p>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 23 (3) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
4 (1) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 5 (-1)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 11
(0) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
3 (1) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 16 (0).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 5 (-1).</p>
</li>
</ul>
<p><a href="https://bugs.launchpad.net/nova/+bug/1832814">1832814: Placement API appears to have issues when compute host
replaced</a> is an
interesting bug. In a switch from RDO to OSA, resource providers are
being duplicated because of a change in node name.</p>
<h1 id="osc-placement">osc-placement</h1>
<p>osc-placement is currently behind by 11 microversions.</p>
<ul>
<li><a href="https://review.opendev.org/666542">https://review.opendev.org/666542</a>
Add support for multiple member_of.</li>
</ul>
<h1 id="main-themes">Main Themes</h1>
<h2 id="nested-magic">Nested Magic</h2>
<p>The overview of the features encapsulated by the term "nested magic"
are in a <a href="https://storyboard.openstack.org/#!/story/2005575">story</a>
and <a href="https://review.opendev.org/662191">spec</a>.</p>
<p>Code related to this:</p>
<ul>
<li>
<p><a href="https://review.opendev.org/663009">https://review.opendev.org/663009</a>
PoC: resourceless request, including some code from
<a href="https://review.opendev.org/657510">https://review.opendev.org/657510</a>
WIP: Allow RequestGroups without resources</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:story/2005575-33753">https://review.opendev.org/#/q/topic:story/2005575-33753</a>
Support for the new root_required query parameter. Introduces the
useful concepts of required wide params and a request wide search
context.</p>
</li>
</ul>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting. A
<a href="https://review.opendev.org/654799">spec</a> has started. There are
some questions about request and response details that need to be
resolved, but the overall concept is sound.</p>
<h2 id="cleanup">Cleanup</h2>
<p>We continue to do cleanup work to lay in reasonable foundations for
the nested work above. As a nice bonus, we keep eking out additional
performance gains too.</p>
<ul>
<li>
<p><a href="https://review.opendev.org/#/q/topic:story/2005443">https://review.opendev.org/#/q/topic:story/2005443</a>
Add a nested-perfload job, using gabbi to create the trees.</p>
<p>Unsurprisingly, nested topologies are slower than non; having
this job will help us track that.</p>
</li>
</ul>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There are five <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And one <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
change</a>.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New discoveries are added to the end. Merged stuff is removed.
Anything that has had no activity in 4 weeks has been removed.</p>
<ul>
<li>
<p><a href="https://review.openstack.org/601596">https://review.openstack.org/601596</a>
Nova: spec: support virtual persistent memory</p>
</li>
<li>
<p><a href="https://review.openstack.org/#/q/topic:bug/1819923">https://review.openstack.org/#/q/topic:bug/1819923</a>
Nova: nova-manage: heal port allocations</p>
</li>
<li>
<p><a href="https://review.openstack.org/650188">https://review.openstack.org/650188</a>
nova-spec: Allow compute nodes to use DISK_GB from shared storage RP</p>
</li>
<li>
<p><a href="https://review.opendev.org/659233">https://review.opendev.org/659233</a>
Cyborg: Placement report</p>
</li>
<li>
<p><a href="https://review.opendev.org/657801">https://review.opendev.org/657801</a>
rpm-packaging: placement service</p>
</li>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: WIP: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/662164">https://review.opendev.org/662164</a>
kolla-ansible: Add a explanatory note for "placement_api_port"</p>
</li>
<li>
<p><a href="https://review.opendev.org/656023">https://review.opendev.org/656023</a>
Nova: Use OpenStack SDK for placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/612497">https://review.opendev.org/612497</a>
Nova: Spec: Provider config YAML file</p>
</li>
<li>
<p><a href="https://review.opendev.org/623558">https://review.opendev.org/623558</a>
Nova: single pass instance info fetch in host manager</p>
</li>
<li>
<p><a href="https://review.opendev.org/634551">https://review.opendev.org/634551</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/665168">https://review.opendev.org/665168</a>
TripleO: Enable Request Filter for Image Types</p>
</li>
<li>
<p><a href="https://review.opendev.org/664689">https://review.opendev.org/664689</a>
Nova: get_ksa_adapter: nix by-service-type confgrp hack</p>
</li>
<li>
<p><a href="https://review.opendev.org/664862">https://review.opendev.org/664862</a>
OSA: Add nova placement to placement migration</p>
</li>
<li>
<p><a href="https://review.opendev.org/657796">https://review.opendev.org/657796</a>
Nova: Defaults missing group_policy to 'none'</p>
</li>
<li>
<p><a href="https://review.opendev.org/666202">https://review.opendev.org/666202</a>
Blazar: Create placement client for each request</p>
</li>
<li>
<p><a href="https://review.opendev.org/666298">https://review.opendev.org/666298</a>
OSA: Fix aio_distro_metal jobs for openSUSE</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>Go outside. Reflect a bit. Then do.</p>More on Maintainership2019-06-14T15:15:00+01:002019-06-14T15:15:00+01:00Chris Denttag:anticdent.org,2019-06-14:/more-on-maintainership.html<p><em>This is a followup to some of the thoughts about being a
"maintainer" raised in <a href="/openstack-denver-summit-reflection.html">OpenStack Denver Summit
Reflection</a>.</em></p>
<p>Of the work that I've done in OpenStack, what has felt most relevant
to me is the work that has not been related to feature development.
In the context of being …</p><p><em>This is a followup to some of the thoughts about being a
"maintainer" raised in <a href="/openstack-denver-summit-reflection.html">OpenStack Denver Summit
Reflection</a>.</em></p>
<p>Of the work that I've done in OpenStack, what has felt most relevant
to me is the work that has not been related to feature development.
In the context of being a "professional maintainer" raised in the
post linked above, this makes a lot of sense.</p>
<p>The work that has felt relevant has improved or optimized either the
existing aspects of the systems themselves, or the process of
creating those systems. For example,
<a href="https://gabbi.readthedocs.io/">Gabbi</a> was created because it was
too difficult to test and understand API changes in the Telemetry
project. Adding support for <a href="https://docs.openstack.org/oslo.config/latest/reference/drivers.html#module-oslo_config.sources._environment">environment variables in
oslo.config</a>
was driven by the <a href="https://anticdent.org/placement-container-playground-9.html">placement container playground
series</a>
which was essentially an exercise in making sure Placement was easy
to test and painless to use in experiments.</p>
<p>The <em>playing around</em> that drove that work has exposed (and fixed)
more bugs and performance issues in Placement, in advance of
common use, than real world use.</p>
<p>More than a year ago I wrote a draft of how I wanted the culture of
the Placement project to be different, once it was extracted from
Nova. Extraction happened near the start of this year. It's
interesting to look back at that draft now.</p>
<p>In it, I discussed why the historical culture of Nova sometimes
seemed "<em>mean</em> in at least three senses of the word: unkind,
miserly, and poor in quality":</p>
<blockquote>
<p>There are many factors that have led to this, but one of them is
the way in which the responsibilities of core reviewers have
evolved to defend stability and limit change while simultaneously
being responsible for creating much of the design and code.</p>
<p>If placement, once extracted, wants to have a different culture it
needs to shrug off this history and actively work to do things
differently. While <em>becoming</em> a core reviewer will probably be
partly the result of creating (and reviewing) features, working on
features should be low down on the priority list once someone is a
core. Priorities should be:</p>
<ol>
<li>Review changes (from others, primarily users)</li>
<li>Fix existing code and tech debt</li>
<li>Refactor and refresh code and documentation</li>
</ol>
<p>Being oriented towards improving existing code rather than
creating features has a few effects:</p>
<ul>
<li>It leaves a clear and inviting doorway for others to participate.</li>
<li>It codifies regular refactoring.</li>
<li>It provides clear safety for incoming new changes to be accepted
based on being "good enough" because there's awareness that fixing
existing code is part of the process.</li>
</ul>
</blockquote>
<p>That sounds a great deal like what I wrote last week, despite having
completely forgotten about the draft from last year: "It probably
means writing less code for features and more code (and docs) to
enable other people to write features."</p>
<p>For that to be possible, a maintenance-oriented person needs
significant amounts of time available for reflection and
experimentation. <em>Headspace</em> in which to be able to think. In my too
long experience as a software professional, very few software
development and/or support teams have recognized the value of
headspace, requiring those who want to reflect and experiment to do
it outside their normal eight hours or not at all. Hours which are
often over-packed with adherence to bastardized and misunderstood
versions of agility, despite clear evidence of the value that comes
from space to think and discover.</p>Placement Update 19-232019-06-14T14:03:00+01:002019-06-14T14:03:00+01:00Chris Denttag:anticdent.org,2019-06-14:/placement-update-19-23.html<p>19-23. I'll be travelling the end of next week so there will be no
19-24.</p>
<h1 id="most-important">Most Important</h1>
<p>We keep having interesting and helpful, but not yet fully
conclusive, discussions related to the <a href="https://review.opendev.org/662191">spec for nested
magic</a>. The discussion on the
spec links back to a few different IRC discussions. Part …</p><p>19-23. I'll be travelling the end of next week so there will be no
19-24.</p>
<h1 id="most-important">Most Important</h1>
<p>We keep having interesting and helpful, but not yet fully
conclusive, discussions related to the <a href="https://review.opendev.org/662191">spec for nested
magic</a>. The discussion on the
spec links back to a few different IRC discussions. Part of the
reason this drags on so much is that we're trying to find a model
that is internally consistent in placement and generally applicable
while satisfying the NUMA requirements from Nova, while not
requiring either Placement or Nova to bend over backwards to get
things right, and while not imposing additional complexity on simple
requests.</p>
<p>(And probably a few other whiles...)</p>
<p>If you have thoughts on these sorts of things, join that review. In
the meantime there are plenty of other things to review (below).</p>
<h1 id="whats-changed">What's Changed</h1>
<ul>
<li>
<p>A blocker migration for incomplete consumers has been added, and
the inline migrations that would guard against the incompleteness
have been removed.</p>
</li>
<li>
<p>CORS configuration and use in Placement has been modernized and
corrected. You can now, if you want, talk to Placement from
JavaScript in your browser. (This was a bug I found while working
on a <a href="https://github.com/cdent/placeview">visualisation toy</a> with
a friend.)</p>
</li>
<li>
<p>Result sets for certain nested provider requests could return
different results in Python versions 2 and 3. This has <a href="https://review.opendev.org/663137">been
fixed</a>.</p>
</li>
<li>
<p>That ^ work was the result of working on implementing <a href="https://docs.openstack.org/placement/latest/specs/train/approved/placement-resource-provider-request-group-mapping-in-allocation-candidates.html">mappings in
allocations</a>
which has <a href="https://docs.openstack.org/placement/latest/placement-api-microversion-history.html#request-group-mappings-in-allocation-candidates">merged today as microversion
1.34</a></p>
</li>
</ul>
<h1 id="specsfeatures">Specs/Features</h1>
<ul>
<li>
<p><a href="https://review.opendev.org/654799">https://review.opendev.org/654799</a>
Support Consumer Types. This has some open questions that need to
be addressed, but we're still go on the general idea.</p>
</li>
<li>
<p><a href="https://review.opendev.org/662191">https://review.opendev.org/662191</a>
Spec for nested magic 1. The easier parts of nested magic:
same_subtree, resourceless request groups, verbose suffixes (already
merged as 1.33). See "Most Important" above.</p>
</li>
</ul>
<p>Some non-placement specs are listed in the Other section below.</p>
<h1 id="storiesbugs">Stories/Bugs</h1>
<p>(Numbers in () are the change since the last pupdate.)</p>
<p>There are 20 (0) stories in <a href="https://storyboard.openstack.org/#!/project_group/placement">the placement
group</a>.
0 (0) are <a href="https://storyboard.openstack.org/#!/worklist/580">untagged</a>.
3 (0) are <a href="https://storyboard.openstack.org/#!/worklist/574">bugs</a>. 6 (2)
are <a href="https://storyboard.openstack.org/#!/worklist/575">cleanups</a>. 11
(0) are <a href="https://storyboard.openstack.org/#!/worklist/594">rfes</a>.
2 (0) are <a href="https://storyboard.openstack.org/#!/worklist/637">docs</a>.</p>
<p>If you're interested in helping out with placement, those stories
are good places to look.</p>
<ul>
<li>
<p>Placement related nova <a href="https://goo.gl/TgiPXb">bugs not yet in progress</a>
on launchpad: 16 (1).</p>
</li>
<li>
<p>Placement related nova <a href="https://goo.gl/vzGGDQ">in progress bugs</a> on
launchpad: 6 (-1).</p>
</li>
</ul>
<p><a href="https://bugs.launchpad.net/nova/+bug/1832814">1832814: Placement API appears to have issues when compute host
replaced</a> is an
interesting bug. In a switch from RDO to OSA, resource providers are
being duplicated because of a change in node name.</p>
<h1 id="osc-placement">osc-placement</h1>
<p>osc-placement is currently behind by 11 microversions.</p>
<p>There are no changes that have had attention in less than 4 weeks.
There are 4 other changes.</p>
<h1 id="main-themes">Main Themes</h1>
<h2 id="nested-magic">Nested Magic</h2>
<p>The overview of the features encapsulated by the term "nested magic"
are in a <a href="https://storyboard.openstack.org/#!/story/2005575">story</a>
and <a href="https://review.opendev.org/662191">spec</a>.</p>
<p>There is some in progress code, mostly WIPs to expose issues and
think about how things ought to work:</p>
<ul>
<li><a href="https://review.opendev.org/663009">https://review.opendev.org/663009</a>
PoC: resourceless request, including some code from
<a href="https://review.opendev.org/657510">https://review.opendev.org/657510</a>
WIP: Allow RequestGroups without resources</li>
</ul>
<h2 id="consumer-types">Consumer Types</h2>
<p>Adding a type to consumers will allow them to be grouped for various
purposes, including quota accounting. A
<a href="https://review.opendev.org/654799">spec</a> has started. There are
some questions about request and response details that need to be
resolved, but the overall concept is sound.</p>
<h2 id="cleanup">Cleanup</h2>
<p>We continue to do cleanup work to lay in reasonable foundations for
the nested work above. As a nice bonus, we keep eking out additional
performance gains too. There are two new stories about some minor
performance degradations:</p>
<ul>
<li>
<p><a href="https://storyboard.openstack.org/#!/story/2005888">https://storyboard.openstack.org/#!/story/2005888</a>
GET /allocations/{non existent consumer} slower than expected</p>
</li>
<li>
<p><a href="https://storyboard.openstack.org/#!/story/2005889">https://storyboard.openstack.org/#!/story/2005889</a>
Allocation mappings have introduced a slight performance reduction</p>
</li>
</ul>
<p>Gibi discovered that osprofiler wasn't working with placement and
then fixed it:</p>
<ul>
<li><a href="https://review.opendev.org/663945">https://review.opendev.org/663945</a>
Add support for osprofiler in wsgi</li>
</ul>
<p>Thanks to Ed Leafe for his report on the <a href="http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007037.html">state of graph database
work</a>.</p>
<h1 id="other-placement">Other Placement</h1>
<p>Miscellaneous changes can be found in <a href="https://review.opendev.org/#/q/project:openstack/placement+status:open">the usual
place</a>.</p>
<p>There are three <a href="https://review.opendev.org/#/q/project:openstack/os-traits+status:open">os-traits
changes</a>
being discussed. And one <a href="https://review.opendev.org/#/q/project:openstack/os-resource-classes+status:open">os-resource-classes
change</a>.</p>
<h1 id="other-service-users">Other Service Users</h1>
<p>New discoveries are added to the end. Merged stuff is removed.
Anything that has had no activity in 4 weeks has been removed.</p>
<ul>
<li>
<p><a href="https://review.openstack.org/601596">https://review.openstack.org/601596</a>
Nova: spec: support virtual persistent memory</p>
</li>
<li>
<p><a href="https://review.openstack.org/#/q/topic:bug/1819923">https://review.openstack.org/#/q/topic:bug/1819923</a>
Nova: nova-manage: heal port allocations</p>
</li>
<li>
<p><a href="https://review.openstack.org/650188">https://review.openstack.org/650188</a>
nova-spec: Allow compute nodes to use DISK_GB from shared storage RP</p>
</li>
<li>
<p><a href="https://review.opendev.org/659233">https://review.opendev.org/659233</a>
Cyborg: Placement report</p>
</li>
<li>
<p><a href="https://review.opendev.org/657801">https://review.opendev.org/657801</a>
rpm-packaging: placement service</p>
</li>
<li>
<p><a href="https://review.opendev.org/657016">https://review.opendev.org/657016</a>
Delete resource providers for all nodes when deleting compute service</p>
</li>
<li>
<p><a href="https://review.opendev.org/654066">https://review.opendev.org/654066</a>
nova test and fix for: Drop source node allocations if finish_resize fails</p>
</li>
<li>
<p><a href="https://review.opendev.org/656885">https://review.opendev.org/656885</a>
nova: WIP: Hey let's support routed networks y'all!</p>
</li>
<li>
<p><a href="https://review.opendev.org/662371">https://review.opendev.org/662371</a>
starlingx: Add placement chart patch to openstack-helm</p>
</li>
<li>
<p><a href="https://review.opendev.org/662229">https://review.opendev.org/662229</a>
helm: WIP: add placement chart</p>
</li>
<li>
<p><a href="https://review.opendev.org/662164">https://review.opendev.org/662164</a>
kolla-ansible: Add a explanatory note for "placement_api_port"</p>
</li>
<li>
<p><a href="https://review.opendev.org/656023">https://review.opendev.org/656023</a>
Nova: Use OpenStack SDK for placement</p>
</li>
<li>
<p><a href="https://review.opendev.org/612497">https://review.opendev.org/612497</a>
Nova: Spec: Provider config YAML file</p>
</li>
<li>
<p><a href="https://review.opendev.org/623558">https://review.opendev.org/623558</a>
Nova: single pass instance info fetch in host manager</p>
</li>
<li>
<p><a href="https://review.opendev.org/663478">https://review.opendev.org/663478</a>
docs: Add Placement service to Minimal deployment for Stein</p>
</li>
<li>
<p><a href="https://review.opendev.org/663270">https://review.opendev.org/663270</a>
devstack: Add setting of placement microversion on tempest conf</p>
</li>
<li>
<p><a href="https://review.opendev.org/634551">https://review.opendev.org/634551</a>
libvirt: report pmem namespaces resources by provider tree</p>
</li>
<li>
<p><a href="https://review.opendev.org/660852">https://review.opendev.org/660852</a>
Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI</p>
</li>
<li>
<p><a href="https://review.opendev.org/661237">https://review.opendev.org/661237</a>
Nova: Validate requested host/node during servers create</p>
</li>
<li>
<p><a href="https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports">https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports</a>
Nova: support move ops with qos ports</p>
</li>
<li>
<p><a href="https://review.opendev.org/664631">https://review.opendev.org/664631</a>
Kolla-ansible: Fix placement log perms in config.json</p>
</li>
<li>
<p><a href="https://review.opendev.org/665168">https://review.opendev.org/665168</a>
TripleO: Enable Request Filter for Image Types</p>
</li>
<li>
<p><a href="https://review.opendev.org/663979">https://review.opendev.org/663979</a>
Neutron: Force segments to use placement 1.1</p>
</li>
<li>
<p><a href="https://review.opendev.org/664689">https://review.opendev.org/664689</a>
Nova: get_ksa_adapter: nix by-service-type confgrp hack</p>
</li>
<li>
<p><a href="https://review.opendev.org/664862">https://review.opendev.org/664862</a>
OSA: Add nova placement to placement migration</p>
</li>
</ul>
<h1 id="end">End</h1>
<p>If you, like me, use this collection of information to drive what to
do early in the next week, you might like
<a href="https://github.com/cdent/placement-reviewme">placement-reviewme</a>
which brute force loads all the links from the HTML version of this
in tabs in your browser.</p>