More robust cleanup of removed hosts for deployment groups

[rohansingh]

Instead of the previous method where we would try to undeploy from hosts
that were no longer registered, instead just apply a straightforward filter
to the removed hosts list.

This is clearer and works without generating a bunch of unnecessary tasks.
Also, the previous version would only work if the set of UP hosts for the
deployment group changed. This version doesn't have that flaw.

This reverts commit 6412fc0.

[rohansingh]

@spotify/helios-team

[mattnworb]

so hosts that are no longer in the DG and happen to be down will stay in the /removed znode until they are deregistered and another rolling-update happens for that group (either manually or because of another host change)?

[rohansingh]

@mattnworb Almost right. The hosts will be removed from the /removed znode when they are deregistered, regardless of another rolling-update happening. This is because we periodically execute the reactor to update the host lists, regardless of rolling updates.

[mattnworb]

thanks for the clarification. I think the stated outcome makes sense. It's hard to evaluate the code on it's own - it is a big bummer that we don't have any real unit tests of any of this rolling-update logic.

[codecov-io]

Current coverage is 51.37% (diff: 92.30%)

Merging #1042 into master will increase coverage by 0.03%

@@             master      #1042   diff @@
==========================================
  Files           275        275          
  Lines         13074      13070     -4   
  Methods           0          0          
  Messages          0          0          
  Branches       1694       1693     -1   
==========================================
+ Hits           6712       6715     +3   
+ Misses         5858       5852     -6   
+ Partials        504        503     -1   

Powered by Codecov. Last update 65eccee...9b744d1

[gimaker]

👍