Skip to content

Match only text fields don't load data correctly when there is a keyword multi field present #134097

@Kubik42

Description

@Kubik42

When index mode is logsdb, and there is a keyword multi field, match_only_text fields cannot load block data.

This happens because match_only_text fields are stored conditionally. However, said condition is not taken into account when we attempt to load block data.

In fact, we actually attempt to load data from stored fields that don't exist. As a result, the following setup returns null for the message field:

Mapping:

            "mappings": {
              "properties": {
                "@timestamp" : {
                  "type": "date"
                },
                "method": {
                  "type": "keyword"
                },
                "message": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "store": true
                    }
                  }
                },
                "ip": {
                  "type": "ip"
                },
                "length": {
                  "type": "long"
                },
                "factor": {
                  "type": "double"
                }
              }
            }

Query:

"query": "FROM $ds | STATS max(length), max(factor) BY message | SORT message | LIMIT 5"

Will return null for the message field.

We can add syntheticSourceDelegate check, similarly to how we do it in TextFieldMapper, however this will only result in #134096.

A better approach imo, is to rework how ignore_above is getting set and expose isIgnoreAboveSet() to minimize any confusion about default values.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions