Remove query-time usage of ByteSequence::slice in PQVectors to reduce object allocations #403
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With #370, I introduced the
ByteSequence::slice
method and started using it on the query path. The PR replaces long lived pq vector ByteSequences with short lived slices of larger ByteSequences created at query time. However, profiling reveals that these objects get created very often for certain workloads, especially brute force workloads in Cassandra. After analyzing the code, it looks trivial to migrate to a solution where we do not create a slice object per access to a pq vector. The main downside is the addition of anoffset
and alength
argument to many methods and the fact that these changes feel brittle, if we want to maintain theslice()
construct, which I think we do for the simplicity of non-query path code. However, given the performance implications and the fact that the interface changes are all internal to jvector, I think this change is worth considering.