Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add /api/docs/:docId/wipeCache endpoint #1486

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fflorent
Copy link
Collaborator

Context

See #1485

Proposed solution

Add an /api/docs/:docId/wipeCache endpoint

This endpoint is the corollary of .../flush: it discards any local copy (the cache) so
the doc worker is forced to fetch the document and metadata from the remote storage.
⚠️ it discard any changes that had not been pushed yet.

This way, when a client open the document again, it is forced to fetch it from the remote storage.

Related issues

Fixes #1485

Has this been tested?

  • 👍 yes, I added tests to the test suite
  • 💭 no, because this PR is a draft and still needs work
  • 🙅 no, because this is not relevant here
  • 🙋 no, because I need help

@fflorent
Copy link
Collaborator Author

If we agree such an endpoint could be useful, the next step will be to implement test for them.

This endpoint is the corollary of .../flush: it discards any local copy (the cache) so
the doc worker is forced to fetch the document and metadata from the remote storage.

Especially useful when changes has been brought to the remote doc storage
and we want to force the doc worker to fetch the version from there.
WARNING: it discard any changes that had not been pushed yet.
@fflorent fflorent force-pushed the wipe-cache-endpoint branch from ea5f6de to 9a7f3b8 Compare February 27, 2025 17:54
// Especially useful when changes has been brought to the remote doc storage
// and we want to force the doc worker to fetch the version from there.
// WARNING: it discard any changes that had not been pushed yet.
this._app.post('/api/docs/:docId/wipeCache', canEdit, throttled(async (req, res) => {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
this._app.post('/api/docs/:docId/wipeCache', canEdit, throttled(async (req, res) => {
this._app.post('/api/docs/:docId/wipeCache', canOwner, throttled(async (req, res) => {

@@ -234,6 +234,10 @@ export class DocStorageManager implements IDocStorageManager {
// nothing to do
}

public async wipeCache(): Promise<void> {
// nothing to do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should throw an error saying it would be a bad idea to wipe a "cache" if it is actually the only copy :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also thinking that checking that a version of the doc exists in S3 could sometimes be life-saving :

  • you're working on a document
  • S3 storage is down, but you can still work on your document (in the background, snapshots don't trigger though)
  • you wipe the "cache"
  • all of your work is gone

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should throw an error saying it would be a bad idea to wipe a "cache" if it is actually the only copy :)

I am not very comfortable with raising an exception, here are the reasons:

  1. A cache may be many things, hence I feel like it is acceptable to just "do nothing" (just like the flush method) ;
  2. this function could be reused elsewhere (I especially think of Remove document cache after a certain inactivity #1494), raising an exception would mean that the caller is aware of the nature of the storage management (local, S3, whatever), the inheritance is convenient for just allowing to ignore the call ;

An alternative I propose is otherwise that the wipeCache method is optional in the IDocStorageManager interface. The caller must check whether the method exists: use optional chaining or a condition + raise an exception if the method is not implemented (something that the endpoint could do).

What do you think?

@paulfitz
Copy link
Member

No objection to the functionality. Some bike-shedding opinions about the name. Cache is ambiguous. Also I'm a little sad about the divergence that has crept in of endpoint naming e.g. "/force-reload" vs "removeUnused" but I think the latter style is winning so we should converge on that. But what do you think about making this functionality be an option on "/force-reload", like "?full=true" or "?clearCache=true" (I think cache is less ambiguous in the context of a document reload).

@vviers vviers assigned vviers and unassigned vviers Feb 28, 2025
@vviers vviers added the gouv.fr label Feb 28, 2025
@vviers
Copy link
Collaborator

vviers commented Feb 28, 2025

But what do you think about making this functionality be an option on "/force-reload", like "?full=true"

I agree that this would be great. Maybe even adding it to the UI in a later PR ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add an endpoint to force the doc worker to fetch again the document from S3
3 participants