[ML] Alerting: Escape URL-like string from being displayed as links

[rbrtj]

Resolves #202507
The PR aims to escape URL-like strings from being displayed as links, mainly in email clients.
To achieve that, we append - hyphens after field_names.

Here are the results of testing for:

• Top records field_name = service.name
• Top influencers influencer_field_name = service.name
• Context message variables: anomaly.entityName = service.name, partition_field_name = testing_it, correlated_by_field_value = correlated_by_field_value, by_field_name = testing.it, by_field_value = test_value. I replaced some field values just because they're expected when constructing full context message, and I'm using mocked data.

I think it covers all occurrences of possible URL-like field names in the alert message.

// Adjusted

• removed = from top influencers
• dropped ( ) from top records
• Added a . dot after the Actual value
  results:

[elasticmachine]

Pinging @elastic/ml-ui (:ml)

[pmuellr]

Some concerns:

• some customers may notice the zero-width char, or it may cause problems in downstream processors
• not sure the matching handles all the cases

Looking at the default message

['{{rule.name}}'] Elastic Stack Machine Learning Alert:
- Job IDs: '{{context.jobIds}}'
- Time: '{{context.timestampIso8601}}'
- Anomaly score: '{{context.score}}'

'{{{context.message}}}'

'{{#context.topInfluencers.length}}'
  Top influencers:
  '{{#context.topInfluencers}}'
    '{{{influencer_field_name}}}' = '{{{influencer_field_value}}}' ['{{score}}']
  '{{/context.topInfluencers}}'
'{{/context.topInfluencers.length}}'

'{{#context.topRecords.length}}'
  Top records:
  '{{#context.topRecords}}'
    '{{function}}'('{{{field_name}}}') '{{{by_field_value}}}''{{{over_field_value}}}''{{{partition_field_value}}}' ['{{score}}']. Typical: '{{typical}}', Actual: '{{actual}}'
  '{{/context.topRecords}}'
'{{/context.topRecords.length}}'

'{{! Replace kibanaBaseUrl if not configured in Kibana }}'
[Open in Anomaly Explorer]('{{{kibanaBaseUrl}}}{{{context.anomalyExplorerUrl}}'})

Don't we really just have a problem with influencer_field_name and field_name? Maybe with the values? But I think the field names are most likely to get invalidly linkified given they are likely to contain . chars.

Did you consider just wrapping these in backtics? `. Markdown and Slack both treat backtics as fixed with font, and I believe exclude them from linkification. So, for example

   `'{{{influencer_field_name}}}'` = '{{{influencer_field_value}}}' ['{{score}}']

I'm curious why there are single quotes around the values. They seem like they'd make the message kinda noisy.

[rbrtj]

Some concerns:

• some customers may notice the zero-width char, or it may cause problems in downstream processors
• not sure the matching handles all the cases

Looking at the default message
Don't we really just have a problem with influencer_field_name and field_name? Maybe with the values? But I think the field names are most likely to get invalidly linkified given they are likely to contain . chars.

Did you consider just wrapping these in backtics? `. Markdown and Slack both treat backtics as fixed with font, and I believe exclude them from linkification. So, for example

   `'{{{influencer_field_name}}}'` = '{{{influencer_field_value}}}' ['{{score}}']

I'm curious why there are single quotes around the values. They seem like they'd make the message kinda noisy.

Hi!
I agree that the initial solution isn't good. There's an additional concern, the fix combines both updating the alert template and adding escaping. This means users who already have rules defined could end up in an intermediate state, having escaping logic but an outdated template, which isn't good.

However, I've tested using backticks, and I don't think they produce good results (gmail client).

For testing, I used influencer_field_name = service.name and influencer_field_value = https://google.com

Example 1:

`{{influencer_field_name}}` = `{{influencer_field_value}}` ['{{score}}']

Example 2:

`{{{influencer_field_name}}}` = `{{{influencer_field_value}}}` ['{{score}}']

[pmuellr]

I think it may be gmail auto-link-ifying, which is something I hadn't thought of.

I sent myself an email with the gmail web ui, with the following text: service.name should not be link-ified, and sure enough, it auto-linkified service.name. So there may not be an easy way around this :-(

[darnautov]

@rbrtj could you please check how these email alerts appear in other email clients, like Apple Mail? If it turns out to be a Gmail specific issue, I agree with @pmuellr, there’s probably not much we can do about it.

[rbrtj]

@pmuellr @darnautov
I have performed some testing with the Apple client, and the results are somewhat different. Given how various mail clients handle linkifying differently, I'm not sure if we should try to fix it, since a solution for one might break something in another.

All testing was done without any escaping logic, only by manipulating the context message markdown.
The message I used: service.name https://google.com google.com www.google.com

The default: {{context.message}}

Triple brackets: {{{context.message}}}

Double brackets with ticks `{{context.message}}`

Triple brackets with ticks `{{{context.message}}}`

It seems like the last one triple brackets with ticks worked well for the apple client.

[pmuellr]

I guess the idea is going to be how to defeat auto-link-ification for field names, like service.name. IIRC, in looking at the context variables used, most were numbers, but the field name is likely going to be problematic due to the .'s in them.

And it seems like "surrounding" these doesn't always work - like with quotes or backtics, etc. I wonder if just suffixing or prefixing the values, at least in the template, might defeat it? So something like field:{{influencer_field_name}} ... (where field: is the "defeating" prefix), or {{influencer_field_name}(field). We don't want to change the actual context value, but it's usage in the mustache template. It would likely have to be something where a string of text without spaces would not be a legal URL, so something like {{influencer_field_name}}?field (not that we would use that!!) wouldn't work, since it would still be a valid URL.

If we find something that works, we could use that in the default context message, and suggest that technique for other folks in the main doc for context variables (and this rule type as well).

[rbrtj]

@pmuellr
Not sure if I understood correctly, but tested for:

field:{{influencer_field_name}} field:{{influencer_field_value}}

{{influencer_field_name}}(field) {{influencer_field_value}}(field)

[pmuellr]

I did some testing, and it appears appending "- " (without the quotes - a hyphen and space) immediately after the variable containing the value service.name will defeat auto-linkification both within our markdown implementation (markdown-it) and gmail.

Obviously not great, but it is good news that there's a common "autolinkify defeater". The text copy will of course be slightly odd looking, but likely better than linkified. If someone clicks the text in the email to select it, they might well pick up the hyphen, but hoping it would be obviously problematic wherever it might be used.

I suspect there are more characters that can be used here. I was hoping : would work since that's kind of a natural for the contextual usage. But it didn't - auto-linkified 😭

I haven't tried with existing full URL values like you show with https://google.com - I suspect the auto-linkifiers may be more forgiving for somethng that REALLY looks like a URL (the prefix) and go ahead and linkify. But maybe that doesn't matter as much as the field names like service.name.

[rbrtj]

Hey, sorry for the late response, but I've tested proposed "- " (just added it to the template, so like {{influencer_field_name}}- ), and it actually seems to break linkifying of all the links, see:

influencer_field_value = 'https://www.google.com';
influencer_field_name = 'service.name';

influencer_field_value = 'www.google.com';
influencer_field_name = 'service.name';

WDYT? @darnautov @pmuellr

[darnautov]

Hey, sorry for the late response, but I've tested proposed "- " (just added it to the template, so like {{influencer_field_name}}- ), and it actually seems to break linkifying of all the links, see:

influencer_field_value = 'https://www.google.com';
influencer_field_name = 'service.name';

``` influencer_field_value = 'www.google.com'; influencer_field_name = 'service.name'; ``` WDYT? @darnautov @pmuellr

I reckon it's fine for influencer_field_name, but not sure if we need to prefix influencer_field_value at all.

[pmuellr]

Ya, I think we want this just for the field NAME and not VALUE. With a - suffix on the field name, hopefully folks will realize the trailing - isn't part of the fiend name, but might be confused if they saw it on a value ("where did the - come from, it's not in my data!).

[rbrtj]

I'm not sure if we're on the same page, but adding a hyphen also breaks valid links.
For the specific case of influencer_field_name = service.name, it would work fine, but I'm assuming both the field name and field value can be valid links. Are we okay with this change?

// Edit: The original case with service.name wouldn't be solved by this solution, since it comes from {{context.message}} and we'd only append the hyphen in the default message skeleton so that users can delete it themselves, right?

[pmuellr]

I'm assuming both the field name and field value can be valid links. Are we okay with this change?

Seems to me like a field name that was a valid URL the user would want to link to, is a bit of a stretch. Like, they had a field named elastic.co, and you'd want that field name to be a link to that? Is there an existing use case for field names that happen to be useful links for the user?

// Edit: The original case with service.name wouldn't be solved by this solution, since it comes from {{context.message}} and we'd only append the hyphen in the default message skeleton so that users can delete it themselves, right?

You could do this in both the constructed context.message variable AND in the default message skeleton supplied in the UX. "This" being whatever we figure out is the "best way to defeat auto-linkification".

[rbrtj]

I'm assuming both the field name and field value can be valid links. Are we okay with this change?

Seems to me like a field name that was a valid URL the user would want to link to, is a bit of a stretch. Like, they had a field named elastic.co, and you'd want that field name to be a link to that? Is there an existing use case for field names that happen to be useful links for the user?

// Edit: The original case with service.name wouldn't be solved by this solution, since it comes from {{context.message}} and we'd only append the hyphen in the default message skeleton so that users can delete it themselves, right?

You could do this in both the constructed context.message variable AND in the default message skeleton supplied in the UX. "This" being whatever we figure out is the "best way to defeat auto-linkification".

Seems to me like a field name that was a valid URL the user would want to link to, is a bit of a stretch, I agree. I've updated the PR with the fix which includes appending hyphens after field_names.

[peteharverson]

Can we lose the = here? I think e.g.

IMO

customer_full_name.keyword- Sultan Al Bryan [4]

looks better than

customer_full_name.keyword- = Sultan Al Bryan [4]

[rbrtj]

I don't have a strong opinion on this, I agree it looks a bit less odd.
WDYT @darnautov?

[darnautov]

agree, makes sense to use = (if it still breaks auto linkifying)

[rbrtj]

Adjusted the message in e7ed992

[peteharverson]

Would dropping the brackets ( and ) improve the formatting here?

high_sum taxful_total_price- 'Wilhemina St. Ryan' [0]. Typical: $63.86, Actual: $200

maybe better than:

high_sum(taxful_total_price-) 'Wilhemina St. Ryan' [0]. Typical: $63.86, Actual: $200

[rbrtj]

Done in e7ed992

[peteharverson]

Tested latest changes with a Slack Connector and an Email connector and LGTM.

While using the - is not ideal, I think the latest formatting edits are as good as we're going to get. Example email:

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: e7ed992

Failed CI Steps

• FTR Configs #51

Test Failures

• [job] [logs] FTR Configs #51 / Reporting Generate CSV from SearchSource validation Searches large amount of data, stops at Max Size Reached

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
ml	5.4MB	5.4MB	+136.0B

History

• 💛 Build #346301 was flaky 03c7be8
• 💚 Build #345758 succeeded b6ec09a
• 💛 Build #326760 was flaky 7c348b6
• 💚 Build #322416 succeeded 33de791

cc @rbrtj

[kibanamachine]

Starting backport for target branches: 9.2

https://github.com/elastic/kibana/actions/runs/18281686119

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	9.2

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation