[HTTP] The Rate-Limiter UX improvements

[dokmic]

Summary

This PR addresses the following items:

• Exclude critical routes from the rate-limiter to serve basic functionality (resolves [Rate Limiting] Exclude critical routes from the rate-limiter #227771).
• Refactor the fatal errors service to make it extensible (resolves [Core] Refactor the fatal errors service to support custom error pages #227776).
• Get rid of enzyme in the fatal errors service (resolves [Unzyme] Remove enzyme dependencies in the fatal errors service #227777).
• Add an error page showing when the server is overloaded (resolves [Rate Limiting] Add an error page to show on the 429 status code #221387).
• Add a response header returning the cooldown timeout (resolves [Rate Limiting] Add a header to the response returned by the rate limiter to identify throttled requests #221386).
• Add support for intercepting the fetch call to provide better extensibility of the HTTP client (resolves [HTTP] Provide a fetch call interceptor in the client-side fetch service #227774).
• Add a mechanism for repeating throttled HTTP requests (resolves [Rate Limiting] Implement a retry mechanism with exponential backoff in the built-in HTTP client to handle 429 status codes #221385).
• Extend the status service to report rate-limiting (resolves Add logging when HTTP rate-limiter starts/ends #212814).

Testing

To test the error page, headers, and the request repetition, please use the following config snippet:

server.rateLimiter.enabled: true
server.rateLimiter.elu: 0.01
server.rateLimiter.term: short

To test the status changes, please increase the threshold value and refresh Kibana a few times:

server.rateLimiter.elu: 0.2

Screenshots

Server is overloaded error page

Error page with a countdown

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[florent-leborgne]

Docs changes LGTM.

Just one thing that I'm not sure of: Will these changes also apply to API users using previous versions like 9.0 or 9.1? If not, it may be good too call out that this change applies from version 9.2 using an availability property as demonstrated on this page. These can also be applied at the parameter and property level when those availabilities are introduced later than the original API-level availability.

Ignore my comment if not relevant for this change :)

[TinaHeiligers]

LGTM

[jloleysens]

Just left a few drive-by comments, overall this looks great! Would be awesome to see a demo of this new error page to get a better sense of the UX.

[elena-shostak]

Security test changes LGTM

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: d0625ee

Failed CI Steps

• FTR Configs #5
• FTR Configs #52
• FTR Configs #52
• FTR Configs #68
• FTR Configs #68
• FTR Configs #109
• FTR Configs #109
• FTR Configs #130
• FTR Configs #130
• Detection Engine - Security Solution Cypress Tests #3
• Detection Engine - Exceptions - Security Solution Cypress Tests #1
• Detection Engine - Exceptions - Security Solution Cypress Tests #3
• Investigations - Security Solution Cypress Tests #2
• Investigations - Security Solution Cypress Tests #3

Test Failures

• [job] [logs] Investigations - Security Solution Cypress Tests #2 / Alert details expandable flyout left panel prevalence should open host preview when click on host name should open host preview when click on host name
• [job] [logs] Investigations - Security Solution Cypress Tests #2 / Alert details expandable flyout left panel prevalence should open user preview when click on user name should open user preview when click on user name
• [job] [logs] Detection Engine - Exceptions - Security Solution Cypress Tests #3 / Auto populate exception with Alert data Should create a Rule exception from Alerts take action button and change multiple exception items without resetting to initial auto-prefilled entries Should create a Rule exception from Alerts take action button and change multiple exception items without resetting to initial auto-prefilled entries
• [job] [logs] Detection Engine - Exceptions - Security Solution Cypress Tests #3 / Auto populate exception with Alert data Should delete all prefilled exception entries when creating a Rule exception from Alerts take action button without resetting to initial auto-prefilled entries Should delete all prefilled exception entries when creating a Rule exception from Alerts take action button without resetting to initial auto-prefilled entries
• [job] [logs] Detection Engine - Security Solution Cypress Tests #3 / bulk fill rule gaps handle the case when the request to fill gaps errors handle the case when the request to fill gaps errors
• [job] [logs] Detection Engine - Exceptions - Security Solution Cypress Tests #1 / Close matching Alerts Should close all alerts from if several rules has shared exception list Should close all alerts from if several rules has shared exception list
• [job] [logs] FTR Configs #68 / Cloud Security Posture Agentless cloud Hyperlink on PostInstallation Modal should have the correct URL
• [job] [logs] FTR Configs #68 / Cloud Security Posture Agentless cloud Hyperlink on PostInstallation Modal should have the correct URL
• [job] [logs] FTR Configs #130 / cloud_security_posture Serverless - Agentless CIS Integration Page Serverless - Agentless CIS_AWS edit flow user should save and edit agentless integration policy
• [job] [logs] FTR Configs #130 / cloud_security_posture Serverless - Agentless CIS Integration Page Serverless - Agentless CIS_AWS edit flow user should save and edit agentless integration policy
• [job] [logs] Detection Engine - Exceptions - Security Solution Cypress Tests #3 / Manage lists from "Shared Exception Lists" page Create/Export/Delete List Create exception list Create exception list
• [job] [logs] Detection Engine - Exceptions - Security Solution Cypress Tests #3 / Manage lists from "Shared Exception Lists" page Create/Export/Delete List Link rules to shared exception list Link rules to shared exception list
• [job] [logs] Investigations - Security Solution Cypress Tests #2 / Multiple indicators Indicator page search should handle all search actions should handle all search actions
• [job] [logs] FTR Configs #109 / serverless observability UI Onboarding Onboarding Auto-Detect guides user through data onboarding
• [job] [logs] FTR Configs #109 / serverless observability UI Onboarding Onboarding Auto-Detect guides user through data onboarding
• [job] [logs] FTR Configs #52 / Synthetics API Tests AddProjectMonitors "before all" hook for "project monitors - saves space as data stream namespace"
• [job] [logs] FTR Configs #52 / Synthetics API Tests AddProjectMonitors "before all" hook for "project monitors - saves space as data stream namespace"

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
core	444	453	+9

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/core-status-common	22	20	-2
@kbn/core-status-server	3	1	-2
@kbn/core-status-server-internal	24	25	+1
total			-3

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
core	476.0KB	478.4KB	+2.3KB

Unknown metric groups

API count

id	before	after	diff
@kbn/core-fatal-errors-browser	9	14	+5
@kbn/core-http-browser	116	120	+4
@kbn/core-metrics-server-internal	6	7	+1
@kbn/core-status-common	33	31	-2
@kbn/core-status-server	20	18	-2
@kbn/core-status-server-internal	24	25	+1
total			+7

ESLint disabled line counts

id	before	after	diff
@kbn/core-fatal-errors-browser-internal	4	3	-1

Total ESLint disabled count

id	before	after	diff
@kbn/core-fatal-errors-browser-internal	4	3	-1

History

• 💚 Build #320401 succeeded 38b37ae
• 💔 Build #320029 failed b5ff319
• 💔 Build #319966 failed 9ebc0d9
• 💔 Build #319706 failed 60e4eef

cc @dokmic

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227678 locally
cc: @dokmic

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227678 locally
cc: @dokmic