Merge _find_coordinator_id methods #2127
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
fyi @swenzel |
jeffwidman
force-pushed
the
merge-_find_coordinator_ids-methods
branch
from
d579afe to
57ccb4d
Compare
Previously there were two methods: * `_find_coordinator_id()` * `_find_many_coordinator_ids()` But they do basically the same thing internally. And we need the plural two places, but the singular only one place. So merge them, and change the function signature to take a list of `group_ids` and return a dict of `group_id: coordinator_id`s. As a result of this, the `describe_groups()` command should scale better because the `_find_coordinator_ids()` command issues all the requests async, instead of sequentially blocking as the `described_groups()` used to do.
jeffwidman
force-pushed
the
merge-_find_coordinator_ids-methods
branch
from
57ccb4d to
80664a5
Compare
gabriel-tincu
pushed a commit
to aiven/kafka-python
that referenced
this pull request
Sep 22, 2020
Previously there were two methods: * `_find_coordinator_id()` * `_find_many_coordinator_ids()` But they do basically the same thing internally. And we need the plural two places, but the singular only one place. So merge them, and change the function signature to take a list of `group_ids` and return a dict of `group_id: coordinator_id`s. As a result of this, the `describe_groups()` command should scale better because the `_find_coordinator_ids()` command issues all the requests async, instead of sequentially blocking as the `described_groups()` used to do.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously there were two methods:
_find_coordinator_id()_find_many_coordinator_ids()But they do basically the same thing internally. And we need the plural
two places, but the singular only one place.
So merge them, and change the function signature to take a list of
group_idsand return a dict ofgroup_id: coordinator_id's.As a result of this, the
describe_groups()command should scale betterbecause the
_find_coordinator_ids()command issues all the requestsasync, instead of sequentially blocking as the
described_groups()usedto do.
IIRC, this deviates slightly from the Java client, as they only take a singular group ID... mostly because it gets tricky to handle retriable vs non-retriable errors when you send a bunch at once. However, we don't handle errors quite the same way--we just raise if the futures don't complete, and also raise if we encounter any problems in the returned
FindCoordinatorResponse. Given that, IMO this is more useful/scalable...Fix #2124
This change is