Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add archive utility #208

Open
wants to merge 18 commits into
base: devel
Choose a base branch
from

Conversation

hagertnl
Copy link
Contributor

Adds the archive utility.

"Archive" does not interface with tape storage archives, but rather refers to a simplified test directory structure that optionally includes/excludes build directory and work directories. From that directory, you may tar/xfer/hsi_xfer those results anywhere, this script is just making that directory structure more amenable to archiving.

@hagertnl hagertnl marked this pull request as ready for review March 24, 2025 14:01
@hagertnl
Copy link
Contributor Author

I tested this briefly this morning on some unit tests (testing a few runs of a failed check, failed build, and a successful test). This is now working, and I've confirmed that the deletion flags are operating correctly. As of now, I'm not aware of any bugs in this script. I do still need to update the docs before merging this, but the main thing that needs review is the utility itself.

Caveats -- the script does not filter out runs that haven't completed. All runs found in the time frame will be archived. If you have deletion flags like --delete-scratch-dir or --delete-run-dir, your actively running tests will fail. You should put a safe --endtime on the query. Running any of these delete options should be handled with extreme caution, as these are the only destructive flags available. I also do not endorse the use of --force, but it works, and it's there. I would strongly recommend handling single conflicting job IDs by hand.

@hagertnl
Copy link
Contributor Author

One thing that's still under consideration is -- should we create a dot file like Run_Archive/<testid>/.archive_log.txt, where we append lines like:

Date                 Options                                    Path
2025-03-03T10:00:00  --keep-build-dir=ALWAYS                    /path/to/my/archive

@hagertnl
Copy link
Contributor Author

As much as a .archive_log.txt might be nice, I think it may be best to pass on it for now. It's possible someone may want to archive tests as one user tests run by another user, and while we could add try/catch to say ".archive_log.txt can't be opened", I think it would be confusing when that file is there sometimes, but not there another. Given that archiving isn't destructive to the original directory by default, I'd say it's fine to continue without this log file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant