cmd/geth:Implement freezer truncation as a subcommand(fixes #31135) #31351
Conversation
|
|
||
| for low <= high { | ||
| mid := (low + high) / 2 | ||
| header := rawdb.ReadHeader(db, rawdb.ReadCanonicalHash(db, mid), mid) | ||
| if header == nil { | ||
| return fmt.Errorf("header %d not found", mid) | ||
| } | ||
|
|
||
| if header.Difficulty.Sign() == 0 { | ||
| // This is a post-merge block, look earlier | ||
| high = mid - 1 | ||
| mergeBlock = mid | ||
| found = true | ||
| } else { | ||
| // This is a pre-merge block, look later | ||
| low = mid + 1 | ||
| } | ||
| } |
There was a problem hiding this comment.
You could do something like this here
| for low <= high { | |
| mid := (low + high) / 2 | |
| header := rawdb.ReadHeader(db, rawdb.ReadCanonicalHash(db, mid), mid) | |
| if header == nil { | |
| return fmt.Errorf("header %d not found", mid) | |
| } | |
| if header.Difficulty.Sign() == 0 { | |
| // This is a post-merge block, look earlier | |
| high = mid - 1 | |
| mergeBlock = mid | |
| found = true | |
| } else { | |
| // This is a pre-merge block, look later | |
| low = mid + 1 | |
| } | |
| } | |
| sort.Search(*headNumber, func(index int) bool { | |
| header := rawdb.ReadHeader(db, rawdb.ReadCanonicalHash(db, index), index) | |
| if header == nil { | |
| panic(fmt.Sprintf("header %d not found", index)) | |
| } | |
| return header.Difficulty.Sign() == 0 | |
| }) |
There was a problem hiding this comment.
Can't we just assume the merge block is hardcoded to ethereum mainnet value? , and assume that we are running on a pre-merged chain?
|
|
||
| // First, read all headers up to the merge block | ||
| log.Info("Reading headers to preserve them", "count", mergeBlock) | ||
| headers := make([][]byte, mergeBlock) |
There was a problem hiding this comment.
So this will be 15 Gbs of memory on mainnet
| } | ||
|
|
||
| // Re-insert the headers and hashes | ||
| log.Info("Re-inserting headers", "count", len(headers)) |
There was a problem hiding this comment.
I think this will not work. Yes you can re-insert the headers, but the freezer has the assumption that all of its tables have the same length. So upon next boot it will truncate the headers to match that of blocks, receipts etc.
There was a problem hiding this comment.
It seems we have two options:
- relax this assumption in the freezer
- Move the headers and hashes to a fresh freezer
There was a problem hiding this comment.
Thankyou for pointing out the issues.
…nfigurable batch size
|
I've updated the PR with optimizations that address all the concerns raised: 1. Memory Usage Issue
2. Freezer Table Length Assumption
3. Hardcoded Merge Block Values
Additional Improvements
Please let me know your thoughts on it. |
…instead of progress reporter
|
Hii @s1na , I fixed the issues you have mentioned and made some modifications to the codebase to make it more readable and maintainable. Can you please review it and share your feedback. |
|
We decided to work on our own version of this. Thank you for your contribution! |
Hii, I have implemented a new subcommand
geth db truncate-freezerthat truncates the freezer at the merge block, keeping headers but removing bodies. This addresses issue #31135.These are the Implementation Details
truncate-freezerto thedbcommandThis implementation follows the same pattern as the existing
prune-historycommand and uses the same underlyingtruncateAncientStoremethod.Fixes #31135