You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/sources/send-data/otel/_index.md
+72-61Lines changed: 72 additions & 61 deletions
Original file line number
Diff line number
Diff line change
@@ -16,14 +16,21 @@ For ingesting logs to Loki using the OpenTelemetry Collector, you must use the [
16
16
17
17
When logs are ingested by Loki using an OpenTelemetry protocol (OTLP) ingestion endpoint, some of the data is stored as [Structured Metadata]({{< relref "../../get-started/labels/structured-metadata" >}}).
18
18
19
+
You must set `allow_structured_metadata` to `true` within your Loki config file. Otherwise, Loki will reject the log payload as malformed.
20
+
21
+
```yaml
22
+
limits_config:
23
+
allow_structured_metadata: true
24
+
```
25
+
19
26
## Configure the OpenTelemetry Collector to write logs into Loki
20
27
21
28
You need to make the following changes to the [OpenTelemetry Collector config](https://opentelemetry.io/docs/collector/configuration/) to write logs to Loki on its OTLP ingestion endpoint.
22
29
23
30
```yaml
24
31
exporters:
25
32
otlphttp:
26
-
endpoint: http://<loki-addr>/otlp
33
+
endpoint: http://<loki-addr>:3100/otlp
27
34
```
28
35
29
36
And enable it in `service.pipelines`:
@@ -50,7 +57,7 @@ exporters:
50
57
otlphttp:
51
58
auth:
52
59
authenticator: basicauth/otlp
53
-
endpoint: http://<loki-addr>/otlp
60
+
endpoint: http://<loki-addr>:3100/otlp
54
61
55
62
service:
56
63
extensions: [basicauth/otlp]
@@ -118,50 +125,52 @@ It currently only supports changing the storage of Attributes. Here is how the c
118
125
119
126
```yaml
120
127
# OTLP log ingestion configurations
121
-
otlp_config:
122
-
# Configuration for Resource Attributes to store them as index labels or
123
-
# Structured Metadata or drop them altogether
124
-
resource_attributes:
125
-
# Configure whether to ignore the default list of resource attributes set in
126
-
# 'distributor.otlp.default_resource_attributes_as_index_labels' to be
127
-
# stored as index labels and only use the given resource attributes config
128
-
[ignore_defaults: <boolean>]
129
-
130
-
[attributes_config: <list of attributes_configs>]
131
-
132
-
# Configuration for Scope Attributes to store them as Structured Metadata or
133
-
# drop them altogether
134
-
[scope_attributes: <list of attributes_configs>]
135
-
136
-
# Configuration for Log Attributes to store them as Structured Metadata or
137
-
# drop them altogether
138
-
[log_attributes: <list of attributes_configs>]
139
-
140
-
attributes_config:
141
-
# Configures action to take on matching Attributes. It allows one of
142
-
# [structured_metadata, drop] for all Attribute types. It additionally allows
143
-
# index_label action for Resource Attributes
144
-
[action: <string> | default = ""]
145
-
146
-
# List of attributes to configure how to store them or drop them altogether
147
-
[attributes: <list of strings>]
148
-
149
-
# Regex to choose attributes to configure how to store them or drop them
150
-
# altogether
151
-
[regex: <Regexp>]
128
+
limits_config:
129
+
otlp_config:
130
+
# Configuration for Resource Attributes to store them as index labels or
131
+
# Structured Metadata or drop them altogether
132
+
resource_attributes:
133
+
# Configure whether to ignore the default list of resource attributes set in
134
+
# 'distributor.otlp.default_resource_attributes_as_index_labels' to be
135
+
# stored as index labels and only use the given resource attributes config
136
+
[ignore_defaults: <boolean>]
137
+
138
+
[attributes_config: <list of attributes_configs>]
139
+
140
+
# Configuration for Scope Attributes to store them as Structured Metadata or
141
+
# drop them altogether
142
+
[scope_attributes: <list of attributes_configs>]
143
+
144
+
# Configuration for Log Attributes to store them as Structured Metadata or
145
+
# drop them altogether
146
+
[log_attributes: <list of attributes_configs>]
147
+
148
+
attributes_config:
149
+
# Configures action to take on matching Attributes. It allows one of
150
+
# [structured_metadata, drop] for all Attribute types. It additionally allows
151
+
# index_label action for Resource Attributes
152
+
[action: <string> | default = ""]
153
+
154
+
# List of attributes to configure how to store them or drop them altogether
155
+
[attributes: <list of strings>]
156
+
157
+
# Regex to choose attributes to configure how to store them or drop them
158
+
# altogether
159
+
[regex: <Regexp>]
152
160
```
153
161
154
162
Here are some example configs to change the default mapping of OTLP to Loki format:
155
163
156
164
#### Example 1:
157
165
158
166
```yaml
159
-
otlp_config:
160
-
resource_attributes:
161
-
attributes_config:
162
-
- action: index_label
163
-
attributes:
164
-
- service.group
167
+
limits_config:
168
+
otlp_config:
169
+
resource_attributes:
170
+
attributes_config:
171
+
- action: index_label
172
+
attributes:
173
+
- service.group
165
174
```
166
175
167
176
With the example config, here is how various kinds of Attributes would be stored:
@@ -172,12 +181,13 @@ With the example config, here is how various kinds of Attributes would be stored
172
181
#### Example 2:
173
182
174
183
```yaml
175
-
otlp_config:
176
-
resource_attributes:
177
-
ignore_defaults: true
178
-
attributes_config:
179
-
- action: index_label
180
-
regex: service.group
184
+
limits_config:
185
+
otlp_config:
186
+
resource_attributes:
187
+
ignore_defaults: true
188
+
attributes_config:
189
+
- action: index_label
190
+
regex: service.group
181
191
```
182
192
183
193
With the example config, here is how various kinds of Attributes would be stored:
@@ -188,25 +198,26 @@ With the example config, here is how various kinds of Attributes would be stored
188
198
#### Example 2:
189
199
190
200
```yaml
191
-
otlp_config:
192
-
resource_attributes:
193
-
attributes_config:
194
-
- action: index_label
195
-
regex: service.group
196
-
scope_attributes:
197
-
- action: drop
198
-
attributes:
199
-
- method.name
200
-
log_attributes:
201
-
- action: structured_metadata
202
-
attributes:
203
-
- user.id
204
-
- action: drop
205
-
regex: .*
201
+
limits_config:
202
+
otlp_config:
203
+
resource_attributes:
204
+
attributes_config:
205
+
- action: index_label
206
+
regex: service.group
207
+
scope_attributes:
208
+
- action: drop
209
+
attributes:
210
+
- method.name
211
+
log_attributes:
212
+
- action: structured_metadata
213
+
attributes:
214
+
- user.id
215
+
- action: drop
216
+
regex: .*
206
217
```
207
218
208
219
With the example config, here is how various kinds of Attributes would be stored:
209
220
* Store all 17 Resource Attributes mentioned earlier and `service.group` Resource Attribute as index labels.
210
221
* Store remaining Resource Attributes as Structured Metadata.
211
222
* Drop Scope Attribute named `method.name` and store all other Scope Attributes as Structured Metadata.
212
-
* Store Log Attribute named `user.id` as Structured Metadata and drop all other Log Attributes.
223
+
* Store Log Attribute named `user.id` as Structured Metadata and drop all other Log Attributes.
0 commit comments