{m365_defender,microsoft_defender_endpoint}.vulnerability: Handle empty response and 403 by kcreddy · Pull Request #15749 · elastic/integrations

kcreddy · 2025-10-24T12:32:55Z

Proposed commit message

{m365_defender,microsoft_defender_endpoint}.vulnerability: Handle empty response and 403 due to expired URLs.

We see cases where the API sends 200 with empty ("") response when 
fetching vulnerabilities using SAS URLs. Since this errored-URL is not 
removed from CEL work_list, it leads to following errors every interval:
"""
failed evaluation: failed eval: ERROR: <input>:2:43: file: EOF
 |   state.?work_list.orValue([]).size() > 0 ?
 | ..........................................^
"""

After a while when the signatures expire (controlled with `sas_valid_hours`), 
following error is noticed:
"""
<?xml version=\"1.0\" encoding=\"utf-8\"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:xxxxxx-xxxx-x-xxx--
Time:2025-10-20T02:07:09.6560520Z</Message><AuthenticationErrorDetail>Signature not valid in the specified key time frame: Key start [Mon, 20 Oct 2025 01:05:29 GMT] - Key expiry [Mon, 20 Oct 2025 02:05:29 GMT] - Current [Mon, 20 Oct 2025 02:07:09 GMT]</AuthenticationErrorDetail></Error>
"""

Handle both these cases in {m365_defender,microsoft_defender_endpoint}.vulnerability 
data stream by adding specific error.message asking users to increase "SAS Valid Hours"
config option. Add troubleshooting section inside the documentation indicating the same.


Also, bring the m365_defender.vulnerability in parity with 
microsoft_defender_endpoint.vulnerability data stream adding oauth endpoint options 
from #15667.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Add/Replace the following files inside packages/m365_defender/_dev/deploy/docker:
download-vulnerability2.log.gz
vulnerability-http-mock-config.yml
Run system tests. eval "$(elastic-package stack shellinit)" && elastic-package test system --generate -v --data-streams=vulnerability
This should produce error.message indicating users to increase SAS Valid Hours option.

Related issues

Closes {m365_defender,microsoft_defender_endpoint}.vulnerability: Handle empty response #15751

elasticmachine · 2025-10-24T13:53:41Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

efd6 · 2025-10-26T21:02:09Z

+                "error": {
+                  "code": string(resp.StatusCode),
+                  "id": string(resp.Status),
+                  "message": decoded_body.decode_error + ". Download URLs are likely expired.Try increasing the 'SAS Valid Hours' parameter to download more data.",


Suggested change

"message": decoded_body.decode_error + ". Download URLs are likely expired.Try increasing the 'SAS Valid Hours' parameter to download more data.",

"message": decoded_body.decode_error + ": Download URLs are likely expired. Try increasing the 'SAS Valid Hours' parameter to download more data.",

efd6 · 2025-10-26T21:07:59Z

+                "error": {
+                  "code": string(resp.StatusCode),
+                  "id": string(resp.Status),
+                  "message": decoded_body.decode_error + ". Download URLs are likely expired.Try increasing the 'SAS Valid Hours' parameter to download more data.",


Suggested change

"message": decoded_body.decode_error + ". Download URLs are likely expired.Try increasing the 'SAS Valid Hours' parameter to download more data.",

"message": decoded_body.decode_error + ": Download URLs are likely expired. Try increasing the 'SAS Valid Hours' parameter to download more data.",

Updated in bf98c76

…-vuln-fixes

elastic-vault-github-plugin-prod · 2025-10-30T04:11:59Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

efd6

nits

The alternative approach would be to check that the body is neither "" nor has a "<" prefix. Though I think that (from what I can make out from the description) will be more complex since the two things happen at different encoding levels in the response (as an aside, why do they use an XML error in a JSON API? or for that matter why explicitly gzip the body when the transport could transparently compress with exactly the same compression scheme ¯\_(ツ)_/¯.

All up, I think this is OK, mod the nits.

elasticmachine · 2025-10-30T05:51:13Z

💚 Build Succeeded

Buildkite Build
Commit: ce2bfc0

History

💚 Build #33415 succeeded 0908e9c
💔 Build #33414 failed bf98c76
💚 Build #33134 succeeded 68562b2
💚 Build #33133 succeeded 696dcb3
💚 Build #33129 succeeded 5a026dd

cc @kcreddy

elastic-vault-github-plugin-prod · 2025-10-30T06:29:11Z

Package m365_defender - 5.1.0 containing this change is available at https://epr.elastic.co/package/m365_defender/5.1.0/

elastic-vault-github-plugin-prod · 2025-10-30T06:29:12Z

Package microsoft_defender_endpoint - 4.1.2 containing this change is available at https://epr.elastic.co/package/microsoft_defender_endpoint/4.1.2/

…ty response and 403 (elastic#15749) {m365_defender,microsoft_defender_endpoint}.vulnerability: Handle empty response and 403 due to expired URLs. We see cases where the API sends 200 with empty ("") response when fetching vulnerabilities using SAS URLs. Since this errored-URL is not removed from CEL work_list, it leads to following errors every interval: """ failed evaluation: failed eval: ERROR: <input>:2:43: file: EOF | state.?work_list.orValue([]).size() > 0 ? | ..........................................^ """ After a while when the signatures expire (controlled with `sas_valid_hours`), following error is noticed: """ <?xml version=\"1.0\" encoding=\"utf-8\"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:xxxxxx-xxxx-x-xxx-- Time:2025-10-20T02:07:09.6560520Z</Message><AuthenticationErrorDetail>Signature not valid in the specified key time frame: Key start [Mon, 20 Oct 2025 01:05:29 GMT] - Key expiry [Mon, 20 Oct 2025 02:05:29 GMT] - Current [Mon, 20 Oct 2025 02:07:09 GMT]</AuthenticationErrorDetail></Error> """ Handle both these cases in {m365_defender,microsoft_defender_endpoint}.vulnerability data stream by adding specific error.message asking users to increase "SAS Valid Hours" config option. Add troubleshooting section inside the documentation indicating the same. Also, bring the m365_defender.vulnerability in parity with microsoft_defender_endpoint.vulnerability data stream adding oauth endpoint options from elastic#15667.

kcreddy added 5 commits October 24, 2025 18:02

Handle empty response and 403 error

baa0786

Add changelog

5a026dd

Add support for OAuth2 Endpoint Params option.

a275511

Add changelog entry

669639a

Handle expired URLs in mde.vulnerability data stream

696dcb3

kcreddy changed the title ~~{microsoft_defender_endpoint, m365_defender}.vulnerability: Handle empty response and 403~~ Oct 24, 2025

Update readme with troubleshooting

68562b2

kcreddy marked this pull request as ready for review October 24, 2025 13:53

kcreddy requested a review from a team as a code owner October 24, 2025 13:53

kcreddy self-assigned this Oct 24, 2025

kcreddy added the documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. label Oct 24, 2025

efd6 reviewed Oct 26, 2025

View reviewed changes

kcreddy added 2 commits October 30, 2025 09:10

Address PR comments

bf98c76

Merge remote-tracking branch 'upstream/main' into mircrosoft-defender…

0908e9c

…-vuln-fixes

kcreddy requested a review from efd6 October 30, 2025 03:43

efd6 approved these changes Oct 30, 2025

View reviewed changes

Comment thread packages/m365_defender/data_stream/vulnerability/agent/stream/cel.yml.hbs Outdated

Comment thread packages/microsoft_defender_endpoint/data_stream/vulnerability/agent/stream/cel.yml.hbs Outdated

nits

ce2bfc0

kcreddy merged commit 77f1765 into elastic:main Oct 30, 2025
8 checks passed

kcreddy deleted the mircrosoft-defender-vuln-fixes branch October 30, 2025 06:11

This was referenced Dec 18, 2025

[Microsoft Defender XDR]: (m365_defender) make OAuth2 endpoint params configurable #16134

Open

Add M365 defender oauth2 params for incident and alert #16628

Merged

a03nikki mentioned this pull request Jan 9, 2026

Add Microsoft defender cloud oauth2 params for assessment #16919

Merged

6 tasks

efd6 mentioned this pull request Feb 24, 2026

microsoft_defender_endpoint.vulnerability: single blob failure abandons all remaining downloads #17559

Closed

Navigation Menu

Uh oh!

{m365_defender,microsoft_defender_endpoint}.vulnerability: Handle empty response and 403 - #15749

{m365_defender,microsoft_defender_endpoint}.vulnerability: Handle empty response and 403#15749
kcreddy merged 9 commits into
elastic:mainfrom
kcreddy:mircrosoft-defender-vuln-fixes

kcreddy commented Oct 24, 2025 •

edited

Loading

elasticmachine commented Oct 24, 2025

Uh oh!

efd6 Oct 26, 2025

kcreddy Oct 30, 2025

Uh oh!

efd6 Oct 26, 2025

kcreddy Oct 30, 2025

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod Bot commented Oct 30, 2025 •

edited

Loading

efd6 left a comment

Uh oh!

Uh oh!

elasticmachine commented Oct 30, 2025

Uh oh!

elastic-vault-github-plugin-prod Bot commented Oct 30, 2025

elastic-vault-github-plugin-prod Bot commented Oct 30, 2025

Labels

3 participants

	"message": decoded_body.decode_error + ". Download URLs are likely expired.Try increasing the 'SAS Valid Hours' parameter to download more data.",
	"message": decoded_body.decode_error + ": Download URLs are likely expired. Try increasing the 'SAS Valid Hours' parameter to download more data.",

Uh oh!

Conversation

kcreddy commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

How to test this PR locally

Related issues

elasticmachine commented Oct 24, 2025

Uh oh!

efd6 Oct 26, 2025

Choose a reason for hiding this comment

kcreddy Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

efd6 Oct 26, 2025

Choose a reason for hiding this comment

kcreddy Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod Bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elasticmachine commented Oct 30, 2025

💚 Build Succeeded

History

Uh oh!

elastic-vault-github-plugin-prod Bot commented Oct 30, 2025

elastic-vault-github-plugin-prod Bot commented Oct 30, 2025

Labels

3 participants

kcreddy commented Oct 24, 2025 •

edited

Loading

elastic-vault-github-plugin-prod Bot commented Oct 30, 2025 •

edited

Loading