Extend default log pattern on server-side to include error information#219940
Extend default log pattern on server-side to include error information#219940maryam-saeidi merged 15 commits intoelastic:mainfrom
Conversation
Yeah logging meta by default would be quite a big change suddenly generating a lot more volume than what users using the pattern layout would be used to. So I think adding We probably want to include the errorConversion and change the default pattern layout for the browser too. https://github.com/elastic/kibana/blob/main/src/core/packages/logging/browser-internal/src/layouts/pattern_layout.ts I noticed there's three DEFAULT_PATTERN constants which is probably unnecessary. We can probably remove the following two:
And just keep: Since the common We'll need to update docs in: |
|
|
@rudolf I updated the PR based on your comment. For the two docs that you mentioned, I just changed the pattern. Let me know if something else needs to be changed as well. |
💔 Build Failed
Failed CI StepsTest Failures
Metrics [docs]Module Count
Public APIs missing comments
Page load bundle
History
|
|
Starting backport for target branches: 8.19 |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
…ormation (#219940) (#221339) # Backport This will backport the following commits from `main` to `8.19`: - [Extend default log pattern on server-side to include error information (#219940)](#219940) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maryam Saeidi","email":"maryam.saeidi@elastic.co"},"sourceCommit":{"committedDate":"2025-05-22T14:57:42Z","message":"Extend default log pattern on server-side to include error information (#219940)\n\n## Release Notes\nKibana logging's pattern layout, used by default for the console\nappender, will now use a new default pattern layout\n`[%date][%level][%logger] %message %error`. This will include the error\nname and stack trace if these were included in the log entry. To opt out\nof this behavior users can omit the `%error` placeholder from their log\npattern config in kibana.yml e.g.:\n```\nlogging:\n appenders:\n console:\n type: console\n layout:\n type: pattern\n pattern: \"[%date][%level][%logger] %message\"\n```\n\n## Summary\n\nPreviously, when we pass the error in meta, the information related to\nstacktrace and error message was not available in console. This PR\nchanged the default pattern to also include error information if it is\nprovided in meta (similar to the way that the logging happens when error\nis directly passed to logger.error).\n\nNew pattern: (added `%error` at the end)\n```\n[%date][%level][%logger] %message %error\n```\n\nHere you can see the difference:\n\nLogger:\n\n```\nserver.logger.error(\n `Unable to create Synthetics monitor ${monitorWithNamespace[ConfigKey.NAME]}`,\n { error: e }\n );\n```\n\n#### Before\n\n\n\n\n#### After\n\n\n\n\n\n### Alternative\nWe could also change the MetaConversion and include this information,\nbut we might have additional meta information which I am not sure if it\nis OK to be logged by default. Let me know if you prefer changing\nMetaConversion instead of adding a new error conversion.\n\n<details>\n<summary>Code changes for MetaConversion</summary>\n\n```\nfunction isError(x: any): x is Error {\n return x instanceof Error;\n}\n\nexport const MetaConversion: Conversion = {\n pattern: /%meta/g,\n convert(record: LogRecord) {\n if (!record.meta) {\n return '';\n }\n const { error, ...rest } = record.meta;\n const metaString = Object.keys(rest).length !== 0 ? JSON.stringify(rest) : '';\n let errorString = '';\n\n if (isError(record.meta?.error)) {\n errorString = record.meta?.error.stack || '';\n }\n\n return [metaString, errorString].filter(Boolean).join(' ');\n },\n};\n```\n</details>\n\nHere is how adjusting meta will look like in this case:\n\n\n","sha":"3d86a175d786ed6f9bc47e0e509cf44defbc66b0","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:version","v9.1.0","v8.19.0"],"title":"Extend default log pattern on server-side to include error information","number":219940,"url":"https://github.com/elastic/kibana/pull/219940","mergeCommit":{"message":"Extend default log pattern on server-side to include error information (#219940)\n\n## Release Notes\nKibana logging's pattern layout, used by default for the console\nappender, will now use a new default pattern layout\n`[%date][%level][%logger] %message %error`. This will include the error\nname and stack trace if these were included in the log entry. To opt out\nof this behavior users can omit the `%error` placeholder from their log\npattern config in kibana.yml e.g.:\n```\nlogging:\n appenders:\n console:\n type: console\n layout:\n type: pattern\n pattern: \"[%date][%level][%logger] %message\"\n```\n\n## Summary\n\nPreviously, when we pass the error in meta, the information related to\nstacktrace and error message was not available in console. This PR\nchanged the default pattern to also include error information if it is\nprovided in meta (similar to the way that the logging happens when error\nis directly passed to logger.error).\n\nNew pattern: (added `%error` at the end)\n```\n[%date][%level][%logger] %message %error\n```\n\nHere you can see the difference:\n\nLogger:\n\n```\nserver.logger.error(\n `Unable to create Synthetics monitor ${monitorWithNamespace[ConfigKey.NAME]}`,\n { error: e }\n );\n```\n\n#### Before\n\n\n\n\n#### After\n\n\n\n\n\n### Alternative\nWe could also change the MetaConversion and include this information,\nbut we might have additional meta information which I am not sure if it\nis OK to be logged by default. Let me know if you prefer changing\nMetaConversion instead of adding a new error conversion.\n\n<details>\n<summary>Code changes for MetaConversion</summary>\n\n```\nfunction isError(x: any): x is Error {\n return x instanceof Error;\n}\n\nexport const MetaConversion: Conversion = {\n pattern: /%meta/g,\n convert(record: LogRecord) {\n if (!record.meta) {\n return '';\n }\n const { error, ...rest } = record.meta;\n const metaString = Object.keys(rest).length !== 0 ? JSON.stringify(rest) : '';\n let errorString = '';\n\n if (isError(record.meta?.error)) {\n errorString = record.meta?.error.stack || '';\n }\n\n return [metaString, errorString].filter(Boolean).join(' ');\n },\n};\n```\n</details>\n\nHere is how adjusting meta will look like in this case:\n\n\n","sha":"3d86a175d786ed6f9bc47e0e509cf44defbc66b0"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/219940","number":219940,"mergeCommit":{"message":"Extend default log pattern on server-side to include error information (#219940)\n\n## Release Notes\nKibana logging's pattern layout, used by default for the console\nappender, will now use a new default pattern layout\n`[%date][%level][%logger] %message %error`. This will include the error\nname and stack trace if these were included in the log entry. To opt out\nof this behavior users can omit the `%error` placeholder from their log\npattern config in kibana.yml e.g.:\n```\nlogging:\n appenders:\n console:\n type: console\n layout:\n type: pattern\n pattern: \"[%date][%level][%logger] %message\"\n```\n\n## Summary\n\nPreviously, when we pass the error in meta, the information related to\nstacktrace and error message was not available in console. This PR\nchanged the default pattern to also include error information if it is\nprovided in meta (similar to the way that the logging happens when error\nis directly passed to logger.error).\n\nNew pattern: (added `%error` at the end)\n```\n[%date][%level][%logger] %message %error\n```\n\nHere you can see the difference:\n\nLogger:\n\n```\nserver.logger.error(\n `Unable to create Synthetics monitor ${monitorWithNamespace[ConfigKey.NAME]}`,\n { error: e }\n );\n```\n\n#### Before\n\n\n\n\n#### After\n\n\n\n\n\n### Alternative\nWe could also change the MetaConversion and include this information,\nbut we might have additional meta information which I am not sure if it\nis OK to be logged by default. Let me know if you prefer changing\nMetaConversion instead of adding a new error conversion.\n\n<details>\n<summary>Code changes for MetaConversion</summary>\n\n```\nfunction isError(x: any): x is Error {\n return x instanceof Error;\n}\n\nexport const MetaConversion: Conversion = {\n pattern: /%meta/g,\n convert(record: LogRecord) {\n if (!record.meta) {\n return '';\n }\n const { error, ...rest } = record.meta;\n const metaString = Object.keys(rest).length !== 0 ? JSON.stringify(rest) : '';\n let errorString = '';\n\n if (isError(record.meta?.error)) {\n errorString = record.meta?.error.stack || '';\n }\n\n return [metaString, errorString].filter(Boolean).join(' ');\n },\n};\n```\n</details>\n\nHere is how adjusting meta will look like in this case:\n\n\n","sha":"3d86a175d786ed6f9bc47e0e509cf44defbc66b0"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
elastic#219940) ## Release Notes Kibana logging's pattern layout, used by default for the console appender, will now use a new default pattern layout `[%date][%level][%logger] %message %error`. This will include the error name and stack trace if these were included in the log entry. To opt out of this behavior users can omit the `%error` placeholder from their log pattern config in kibana.yml e.g.: ``` logging: appenders: console: type: console layout: type: pattern pattern: "[%date][%level][%logger] %message" ``` ## Summary Previously, when we pass the error in meta, the information related to stacktrace and error message was not available in console. This PR changed the default pattern to also include error information if it is provided in meta (similar to the way that the logging happens when error is directly passed to logger.error). New pattern: (added `%error` at the end) ``` [%date][%level][%logger] %message %error ``` Here you can see the difference: Logger: ``` server.logger.error( `Unable to create Synthetics monitor ${monitorWithNamespace[ConfigKey.NAME]}`, { error: e } ); ``` #### Before  #### After  ### Alternative We could also change the MetaConversion and include this information, but we might have additional meta information which I am not sure if it is OK to be logged by default. Let me know if you prefer changing MetaConversion instead of adding a new error conversion. <details> <summary>Code changes for MetaConversion</summary> ``` function isError(x: any): x is Error { return x instanceof Error; } export const MetaConversion: Conversion = { pattern: /%meta/g, convert(record: LogRecord) { if (!record.meta) { return ''; } const { error, ...rest } = record.meta; const metaString = Object.keys(rest).length !== 0 ? JSON.stringify(rest) : ''; let errorString = ''; if (isError(record.meta?.error)) { errorString = record.meta?.error.stack || ''; } return [metaString, errorString].filter(Boolean).join(' '); }, }; ``` </details> Here is how adjusting meta will look like in this case: 
Closes elastic/observability-dev#4432 ## Summary Auditing [log levels](https://docs.elastic.dev/kibana-dev-docs/services/logging#log-level) in the Synthetics plugin. ### Example Throwing error when creating a new monitor: <img src="https://github.com/user-attachments/assets/7cead63b-c209-4174-a4f7-7dfad40aa34d" width=500 /> #### Before ``` 1. [2025-04-23T12:29:37.594+02:00][ERROR][plugins.synthetics] Error: newMonitorPromise failed! at /Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:82:15 at async AddEditMonitorAPI.syncNewMonitor (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:88:72) at async handler (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor.ts:123:11) 2. [2025-04-23T12:29:37.595+02:00][ERROR][plugins.synthetics] Unable to create Synthetics monitor https://www.elastic.co/ 3. [2025-04-23T12:29:38.758+02:00][ERROR][plugins.synthetics] Error: newMonitorPromise failed! at /Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:82:15 at async AddEditMonitorAPI.syncNewMonitor (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:88:72) at async handler (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor.ts:123:11) 4. [2025-04-23T12:29:38.760+02:00][ERROR][plugins.synthetics] Unable to create synthetics monitor ``` #### After ``` [2025-05-12T16:06:23.160+02:00][ERROR][plugins.synthetics] Unable to create Synthetics monitor with name https://www.elastic.co/ Error: newMonitorPromise failed! at /Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:82:15 at async AddEditMonitorAPI.syncNewMonitor (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:88:72) at async handler (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor.ts:123:11) ``` #####⚠️ Note After merging this [PR](#219940), we will also have stacktrace locally. ### ❓ Questions 1. When calling `synthetics/params` API with wrong parameters, we get `[2025-05-13T11:05:52.401+02:00][ERROR][http] 400 Bad Request`, which path is responsible for this error? (Since it is from `http`, so it is not logged in synthetics. I am wondering where the validation is happening in this case) **Answer**: It was here: https://github.com/elastic/kibana/blob/main/src/core/packages/http/router-server-internal/src/route.ts#L124
Closes elastic/observability-dev#4432 ## Summary Auditing [log levels](https://docs.elastic.dev/kibana-dev-docs/services/logging#log-level) in the Synthetics plugin. ### Example Throwing error when creating a new monitor: <img src="https://github.com/user-attachments/assets/7cead63b-c209-4174-a4f7-7dfad40aa34d" width=500 /> #### Before ``` 1. [2025-04-23T12:29:37.594+02:00][ERROR][plugins.synthetics] Error: newMonitorPromise failed! at /Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:82:15 at async AddEditMonitorAPI.syncNewMonitor (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:88:72) at async handler (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor.ts:123:11) 2. [2025-04-23T12:29:37.595+02:00][ERROR][plugins.synthetics] Unable to create Synthetics monitor https://www.elastic.co/ 3. [2025-04-23T12:29:38.758+02:00][ERROR][plugins.synthetics] Error: newMonitorPromise failed! at /Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:82:15 at async AddEditMonitorAPI.syncNewMonitor (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:88:72) at async handler (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor.ts:123:11) 4. [2025-04-23T12:29:38.760+02:00][ERROR][plugins.synthetics] Unable to create synthetics monitor ``` #### After ``` [2025-05-12T16:06:23.160+02:00][ERROR][plugins.synthetics] Unable to create Synthetics monitor with name https://www.elastic.co/ Error: newMonitorPromise failed! at /Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:82:15 at async AddEditMonitorAPI.syncNewMonitor (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:88:72) at async handler (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor.ts:123:11) ``` #####⚠️ Note After merging this [PR](elastic#219940), we will also have stacktrace locally. ### ❓ Questions 1. When calling `synthetics/params` API with wrong parameters, we get `[2025-05-13T11:05:52.401+02:00][ERROR][http] 400 Bad Request`, which path is responsible for this error? (Since it is from `http`, so it is not logged in synthetics. I am wondering where the validation is happening in this case) **Answer**: It was here: https://github.com/elastic/kibana/blob/main/src/core/packages/http/router-server-internal/src/route.ts#L124 (cherry picked from commit 3279797) # Conflicts: # x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/bulk_cruds/edit_monitor_bulk.ts # x-pack/solutions/observability/plugins/synthetics/server/routes/suggestions/route.ts
Closes elastic/observability-dev#4432 ## Summary Auditing [log levels](https://docs.elastic.dev/kibana-dev-docs/services/logging#log-level) in the Synthetics plugin. ### Example Throwing error when creating a new monitor: <img src="https://github.com/user-attachments/assets/7cead63b-c209-4174-a4f7-7dfad40aa34d" width=500 /> #### Before ``` 1. [2025-04-23T12:29:37.594+02:00][ERROR][plugins.synthetics] Error: newMonitorPromise failed! at /Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:82:15 at async AddEditMonitorAPI.syncNewMonitor (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:88:72) at async handler (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor.ts:123:11) 2. [2025-04-23T12:29:37.595+02:00][ERROR][plugins.synthetics] Unable to create Synthetics monitor https://www.elastic.co/ 3. [2025-04-23T12:29:38.758+02:00][ERROR][plugins.synthetics] Error: newMonitorPromise failed! at /Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:82:15 at async AddEditMonitorAPI.syncNewMonitor (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:88:72) at async handler (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor.ts:123:11) 4. [2025-04-23T12:29:38.760+02:00][ERROR][plugins.synthetics] Unable to create synthetics monitor ``` #### After ``` [2025-05-12T16:06:23.160+02:00][ERROR][plugins.synthetics] Unable to create Synthetics monitor with name https://www.elastic.co/ Error: newMonitorPromise failed! at /Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:82:15 at async AddEditMonitorAPI.syncNewMonitor (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor/add_monitor_api.ts:88:72) at async handler (/Users/maryam.saeidi/WebstormProjects/kibana-mary/x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/add_monitor.ts:123:11) ``` #####⚠️ Note After merging this [PR](elastic#219940), we will also have stacktrace locally. ### ❓ Questions 1. When calling `synthetics/params` API with wrong parameters, we get `[2025-05-13T11:05:52.401+02:00][ERROR][http] 400 Bad Request`, which path is responsible for this error? (Since it is from `http`, so it is not logged in synthetics. I am wondering where the validation is happening in this case) **Answer**: It was here: https://github.com/elastic/kibana/blob/main/src/core/packages/http/router-server-internal/src/route.ts#L124
|
Hi @maryam-saeidi. Can you add the correct team or feature label to this PR so it gets pulled into the correct release notes? Thank you! |
|
@florent-leborgne I've added "Feature:Logging" label, please let me know if something else is needed :) |
Release Notes
Kibana logging's pattern layout, used by default for the console appender, will now use a new default pattern layout
[%date][%level][%logger] %message %error. This will include the error name and stack trace if these were included in the log entry. To opt out of this behavior users can omit the%errorplaceholder from their log pattern config in kibana.yml e.g.:Summary
Previously, when we pass the error in meta, the information related to stacktrace and error message was not available in console. This PR changed the default pattern to also include error information if it is provided in meta (similar to the way that the logging happens when error is directly passed to logger.error).
New pattern: (added
%errorat the end)Here you can see the difference:
Logger:
Before
After
Alternative
We could also change the MetaConversion and include this information, but we might have additional meta information which I am not sure if it is OK to be logged by default. Let me know if you prefer changing MetaConversion instead of adding a new error conversion.
Code changes for MetaConversion
Here is how adjusting meta will look like in this case: