Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ongoing: Caching of setup-environment-commands for faster initialisation #290

Open
4 of 5 tasks
balupton opened this issue Feb 21, 2025 · 5 comments
Open
4 of 5 tasks
Labels
ongoing Ongoing efforts of incremental improvements

Comments

@balupton
Copy link
Member

balupton commented Feb 21, 2025

You can place this at the start of your config/environment.bash User Configuration file to enable caching:

https://gist.github.com/balupton/4d1af347074e676ff9e5eb96c1cbaf3d#file-environment-bash

Feel free to share your modifications and enhancements below.

Thanks to @joelmccracken for the suggestion, and @molleweide for the brainstorming and initial experimentation.

User configuration is the correct location for this, as caching always incurs risks, especially for something as important and foundational as setup-environment-commands and its role in the shell's init process. As such, having it inside the user configuration allows the user to adapt the behaviour to their own risk management, while also doing the importance of two of Dorothy's principles: convergence to find supremacy, and daily driving to find reliability.

Edge cases and other considerations:

  • Sometimes Dorothy itself recalls sources/environment.* to reload environment variables after say a system modification (such as after setup-util-devel), caching would need to play well with that. Possible improvement here is limiting this to login shell execution, or something like that.
  • If one uses a terminal multiplexer or just has a cmd+t trigger finger, they would initialise several new login shell instances right away, in which case corruption could occur as they are all going through this caching process at once. flock or some other semlock functionality could be used to avoid this. The latest version solves this.
  • If the cache result is outdated, this does not support eventual correctment in this terminal instance. A user config interactive.* modification could be provided for reloading sources/environment.* after a delay or as an alias invocation. The latest version solves this.
  • The above approach should eliminate removed env vars from persisting in eternity, thanks to the comparison of envs before doing cache, however in practice that is still theoretical. The latest version guarantees this.
  • It might need a __mkdirp call, and perhaps $DOROTHY/state is a better location. The latest version solves this.
@balupton balupton added the ongoing Ongoing efforts of incremental improvements label Feb 21, 2025
@balupton
Copy link
Member Author

The latest version implements semlock to prevent corruption.

@balupton
Copy link
Member Author

The latest version implements cache invalidation after a custom duration.

@balupton
Copy link
Member Author

balupton commented Feb 22, 2025

I believe the current version is good enough to go into core after devilbird is merged, where I can:

  • introduce a setup-environment-commands --invalidate-cache before source "$DOROTHY/sources/environment.bash" statements to solve the remaining edge case.
  • have it opt in by the user's config/environment.bash setting a CACHE_VALIDITY_SECONDS variable.

@balupton
Copy link
Member Author

Updated again, this time to use checksum to support more env contexts and to remove the need for the before/context file.

@balupton balupton added this to the Post Launch [v4] milestone Feb 23, 2025
@balupton
Copy link
Member Author

Other things to consider caching as well:

  • styles.bash, as echo-style is slow, using it as a function is faster, so it seems it is styles.bash that is slow
  • get-terminal-theme

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ongoing Ongoing efforts of incremental improvements
Development

No branches or pull requests

1 participant