This bug is a focused version of #41046.
From the append processor documentation:
The append processor appends one or more values to an existing array if the target field already exists and it is an array.
Given a configuration like:
filebeat.inputs:
- type: filestream
id: foo
paths:
- /tmp/flog.log
tags:
- "forwarded"
- "another-tag-from-the-tags-field"
processors:
- append:
target_field: tags
values:
- foo_bar
- add_host_metadata:
when.not.contains.tags: forwarded
output.console:
enabled: true
pretty: true
The add_host_metadata always runs. When looking at the event tags in Kibana, they look like the expected array: [forwarded, another-tag-from-the-tags-field, foo_bar].
The issue:
When tags are added using the input-level tags field, they get added to the event as a []string, then when the append processor runs:
|
// get the existing value of target field |
|
targetVal, err := event.GetValue(target) |
|
if err != nil { |
|
f.logger.Debugf("could not fetch value for key: '%s'. Therefore, all the values will be appended in a new key %s.", target, target) |
|
} else { |
|
targetArr, ok := targetVal.([]interface{}) |
|
if ok { |
|
arr = append(arr, targetArr...) |
|
} else { |
|
arr = append(arr, targetVal) |
|
} |
|
} |
targetVal is []string, so on L 106, the conversion fails (ok: false), so the final result is tags: [["forwarded", "another-tag-from-the-tags-field"], "foo_bar"]
Then when the condition for the add_host_metadata runs, it looks for the string "forwarded", because the search expects an array of strings (not an array of objects), the search fails, therefore the add_host_metadata processor runs.
This bug is a focused version of #41046.
From the append processor documentation:
Given a configuration like:
The
add_host_metadataalways runs. When looking at the event tags in Kibana, they look like the expected array:[forwarded, another-tag-from-the-tags-field, foo_bar].The issue:
When tags are added using the input-level
tagsfield, they get added to the event as a[]string, then when the append processor runs:beats/libbeat/processors/actions/append.go
Lines 101 to 112 in 223b974
targetValis[]string, so on L 106, the conversion fails (ok: false), so the final result istags: [["forwarded", "another-tag-from-the-tags-field"], "foo_bar"]Then when the condition for the
add_host_metadataruns, it looks for the string "forwarded", because the search expects an array of strings (not an array of objects), the search fails, therefore theadd_host_metadataprocessor runs.