detect/entropy: Add entropy keyword #12722
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Issue: 4162 This commit adds a floating-point analog to `uint.rs` targeted initially to be used by the forthcoming entropy keyword.
This commit adds
- Parser for the entropy keyword
- Calculation of content the Shannon entropy value
Issue: 4162
The entropy keyword syntax is the keyword entropy followed by options
and the entropy value for comparison.
The minimum entropy keyword specification is:
entropy: value <entropy-spec>
This results in the calculated entropy value being compared with
<entropy-spec> with the equality operator.
Calculated entropy values are between 0.0 and 8.0, inclusive.
A match occurs when the values and operator agree. This example matches
if the calculated and entropy value are the same.
When entropy keyword options are specified, all options and "value" must
be comma-separated. Options and value may be specified in any order.
Options have default values:
- bytes is equal to the current content length
- offset is 0
- comparison with value is equality
entropy: [bytes <byteval>] [offset <offsetval>] value <entropy-spec>
Using default values:
entropy: bytes 0, offset 0, value =<entropy-spec>
<entropy-spec> is: <operator> (see below) and a value, e.g., "< 4.1"
The following operators are available from the float crate:
- = (default): Match when calculated entropy value equals specified entropy value
- < Match when calculated entropy value is strictly less than specified entropy value
- <= Match when calculated entropy value is less than or equal to specified entropy value
- > Match when calculated entropy value is strictly greater than specified entropy value
- >= Match when calculated entropy value is greater than or equal to specified entropy value
- != Match when calculated entropy value is not equal to specified entropy value
- x-y Match when calculated entropy value is in the range, exclusive
- !x-y Match when calculated entropy value is not in the range, exclusive
This commit adds keyword/build support for the entropy keyword. The entropy keyword compares an entropy value with a value calculated according to the Shannon entropy on the available content. Issue: 4162
This commit causes the content inspection engine to recognize and invoke the entropy "match" function when the entropy keyword is used. Issue: 4162
This commit updates the - Upgrade notes for 7 to 8 - Payload keyword section Both are update to document the new entropy keyword.
jlucovsky
requested review from
jasonish,
jufajardini and
victorjulien
as code owners
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #12722 +/- ##
==========================================
+ Coverage 80.76% 80.80% +0.03%
==========================================
Files 936 939 +3
Lines 259724 260393 +669
==========================================
+ Hits 209765 210402 +637
- Misses 49959 49991 +32
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
Information: QA ran without warnings. Pipeline 25006 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Continuation of #12721
The entropy keyword syntax is the keyword
entropyfollowed by optionsand the entropy value for comparison.
The minimum entropy keyword specification is:
entropy: value <entropy-spec>This results in the calculated entropy value being compared with
with the equality operator.
A match occurs when the values and operator agree. This example matches
if the calculated and entropy value are the same.
When entropy keyword options are specified, all options and "value" must
be comma-separated. Options and value may be specified in any order.
Options have default values:
entropy: [bytes <byteval>] [offset <offsetval>] value <entropy-spec>Using default values:
entropy: bytes 0, offset 0, value =<entropy-spec>is: (see below) and a value, e.g., "< 4.1"
The following operators are available from the float crate introduced with this pr:
- = (default): Match when calculated entropy value equals specified entropy value
- < Match when calculated entropy value is strictly less than specified entropy value
- <= Match when calculated entropy value is less than or equal to the specified entropy value
- > Match when calculated entropy value is strictly greater than specified entropy value
- >= Match when calculated entropy value is greater than or equal to the specified entropy value
- != Match when the calculated entropy value is not equal to the specified entropy value
- x-y Match when calculated entropy value is in the range, exclusive
- !x-y Match when calculated entropy value is not in the range, exclusive
Link to ticket: https://redmine.openinfosecfoundation.org/issues/4162
Describe changes:
Updates:
Provide values to any of the below to override the defaults.
link to the pull request in the respective
_BRANCHvariable.SV_BRANCH=OISF/suricata-verify#2232