-
Notifications
You must be signed in to change notification settings - Fork 634
[AbuseCH] Add Commercial API support for the ThreatFox data stream #20134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
dc62db4
42c35bb
1e37f63
d7ace9d
22d993d
8ce4362
00f5a3f
5e4c42a
a17330c
a138839
9e8902a
4ba3db9
e864342
8d86569
e6e6105
d47d898
f98d126
8eb4747
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,17 @@ | ||
| # newer versions go on top | ||
| - version: "5.0.0" | ||
| changes: | ||
| - description: Emit a configuration error when the Auth Key is not set for the Malware URLs and Malware payloads datasets. | ||
| type: bugfix | ||
| link: https://github.com/elastic/integrations/pull/20134 | ||
| - description: Add Commercial API support for the ThreatFox data stream. | ||
|
moxarth-rathod marked this conversation as resolved.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Severity: 🟠 High MalwareBazaar's api_type/username/password move from data-stream scope to policy-template scope, so 4.2.0 commercial policies lose those values on upgrade - add a breaking-change entry (or var migrate_from) covering it. DetailsVersion 4.2.0 (already on main, PR 20199) shipped api_type, username and password as vars of the malwarebazaar data stream. This PR deletes them from packages/ti_abusech/data_stream/malwarebazaar/manifest.yml and re-declares them on the policy-template input in the root manifest. Fleet does not carry values across that scope change on its own, so an existing MalwareBazaar Commercial API policy upgrading 4.2.0 -> 4.3.0 falls back to the new policy-level default api_type=community with no auth_key configured, and collection breaks until the user re-enters the settings. The 4.3.0 changelog contains only a single Recommendation: Document the migration explicitly, following the precedent set by the 4.0.0 entry in this same file: - version: "4.3.0"
changes:
- description: Add Commercial API support for the ThreatFox data stream.
type: enhancement
link: https://github.com/elastic/integrations/pull/20134
- description: |
The MalwareBazaar Commercial API settings (API Type, Username, Password) moved from the
MalwareBazaar data stream settings to the integration-level settings. Users who configured
the MalwareBazaar Commercial API on 4.2.0 must re-enter API Type, Username and Password
after upgrading.
type: breaking-change
link: https://github.com/elastic/integrations/pull/20134Alternatively, let Fleet carry the values over automatically by bumping - name: api_type
type: select
title: API Type
migrate_from:
scope: stream
stream: malwarebazaar🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @moxarth-rathod, we need breaking-change for this.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||
| type: enhancement | ||
| link: https://github.com/elastic/integrations/pull/20134 | ||
|
efd6 marked this conversation as resolved.
efd6 marked this conversation as resolved.
|
||
| - description: | | ||
| Move API Type, Username, and Password from the MalwareBazaar data stream to the shared integration input so one Spamhaus credential pair covers all commercial datasets. | ||
| Users who configured MalwareBazaar Commercial API on 4.2.0 must re-enter API Type, Username, and Password at the integration level after upgrade. See the Troubleshooting section in the README. | ||
| type: breaking-change | ||
| link: https://github.com/elastic/integrations/pull/20134 | ||
| - version: "4.2.0" | ||
| changes: | ||
| - description: Add Commercial API support for the MalwareBazaar data stream. | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Severity: 🟡 Medium
confidence: mediumpath: packages/ti_abusech/_dev/build/docs/README.md:112The v5.0.0 upgrade note tells 4.2.0 commercial users to re-enter credentials but not to reset the MalwareBazaar URL, which they were told to point at api.spamhaus.com and which is now hidden; add that step.
Details
In 4.2.0 the MalwareBazaar url variable was documented as 'For the Commercial API, use the API base URL (for example https://api.spamhaus.com)', so commercial users were instructed to change it. This PR narrows that variable back to 'URL of the MalwareBazaar Community API', flips it to show_user: false, and routes the commercial base URL through the new package-level commercial_url variable instead. Fleet retains the previously configured url value on upgrade, so a 4.2.0 commercial MalwareBazaar user lands on 5.0.0 with api_type back at community and url still set to https://api.spamhaus.com; the community branch then POSTs to that host and fails. Because url is now show_user: false it is only reachable under Advanced options, so the fix is not discoverable from the steps as written.
Recommendation:
Add the URL reset to the v5.0.0 upgrade steps in
_dev/build/docs/README.md(and regeneratedocs/README.md):🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@moxarth-rathod, this is worth doing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing this out.