Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove query-time usage of ByteSequence::slice in PQVectors to reduce object allocations #403

Conversation

michaeljmarshall
Copy link
Member

With #370, I introduced the ByteSequence::slice method and started using it on the query path. The PR replaces long lived pq vector ByteSequences with short lived slices of larger ByteSequences created at query time. However, profiling reveals that these objects get created very often for certain workloads, especially brute force workloads in Cassandra. After analyzing the code, it looks trivial to migrate to a solution where we do not create a slice object per access to a pq vector. The main downside is the addition of an offset and a length argument to many methods and the fact that these changes feel brittle, if we want to maintain the slice() construct, which I think we do for the simplicity of non-query path code. However, given the performance implications and the fact that the interface changes are all internal to jvector, I think this change is worth considering.

@michaeljmarshall
Copy link
Member Author

I keep seeing failures for:

Test2DThreshold.testThreshold20k:44->testThreshold:74 visited 75.38348% of the vectors, which is more than 75.0%

This doesn't seem related and feels potentially flaky since the tests pass on retry.

@marianotepper marianotepper requested review from jkni and removed request for jbellis and jkni March 20, 2025 21:14
@jkni
Copy link
Collaborator

jkni commented Mar 21, 2025

Looks reasonable to me -- I agree that this is a bit brittle and suggests that we might want to revisit the API in the future, but I wouldn't call it disproportionately awkward/brittle compared to other performance-based sacrifices we've made around these code paths.

@marianotepper
Copy link
Collaborator

I agree the comments above and the PR looks good to me. I suggest we open an issue about the interface so that we do not forget.

michaeljmarshall added a commit to datastax/cassandra that referenced this pull request Mar 21, 2025
@marianotepper marianotepper merged commit 0962ddb into datastax:main Mar 21, 2025
6 checks passed
@michaeljmarshall michaeljmarshall deleted the reduce-bytesequence-slice-usage-on-query-path branch March 21, 2025 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants