Skip to content

[ResponseOps] OAuth2 for webhook connector#218442

Merged
jcger merged 71 commits intoelastic:mainfrom
guskovaue:OAuth2-for-webhook-connector
Sep 19, 2025
Merged

[ResponseOps] OAuth2 for webhook connector#218442
jcger merged 71 commits intoelastic:mainfrom
guskovaue:OAuth2-for-webhook-connector

Conversation

@guskovaue
Copy link
Copy Markdown
Contributor

@guskovaue guskovaue commented Apr 16, 2025

Summary

Closes #216140

We added an extra field to the OAuth 2 form called additionalFields. Even though the RFC doesn't define required parameters other than the client id and the client secret, there might be OAuth providers that require more information.

QA

Token service configuration

We'll use Keycloak as our token service. From Wikipedia:

Keycloak is an open-source software product to allow single sign-on with identity and access management aimed at modern applications and services. 
...
Keycloak supports various protocols such as OpenID, OAuth version 2.0 and SAML and provides features such as user management, two-factor authentication, permissions and roles management, creating token services, etc.

Step 1: Run Keycloak using Docker

Open your terminal and run the following command:

docker run -p 8080:8080 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=admin quay.io/keycloak/keycloak:latest start-dev

Step 2: Configure Keycloak

  1. Access the Admin Console:
  • Open your web browser and go to ⁠http://localhost:8080.
  • Click on the Administration Console link.
  • Log in with the credentials you set in the Docker command:
    • Username: ⁠admin
    • Password: ⁠admin
  1. Create a New Realm:
  • The ⁠master realm is for managing Keycloak itself. You should create a new one for your applications.
  • In the left menu, click on Manage realms > Create Realm.
  • Realm name: ⁠kibana-test
  • Click Create.
  1. Create a Client:
  • A "Client" in Keycloak is an entity that can request authentication, in our case, the Connector.
  • Ensure you are in your ⁠kibana-test realm.
  • In the left menu, click on Clients > Create client.
  • Client ID: ⁠kibana-webhook-connector.
  • Click Next.
  1. Configure Client Authentication:
  • On the next screen, enable the Client authentication toggle.
  • Leave Standard flow on and enable Service accounts role (this is client credentials mode)
  • Click Next & Save
  1. Get Your Credentials and Token URL:
  • After saving, you'll be on the client's configuration page.
  • Go to the Credentials tab.
  • You will see the Client secret. Copy this value.
  • Now, you need the Token URL. Go to the Realm settings in the left menu.
  • On the General tab, click the OpenID Endpoint Configuration link.
  • Copy the token_endpoint. It should be something like http://localhost:8080/realms/kibana-test/protocol/openid-connect/token Make sure the right realm name is in it, like kibana-test and not master (yes, it happened...)

You now have the information needed:

  • Client ID: ⁠kibana-webhook-connector
  • Client Secret: The secret you copied from the Credentials tab.
  • Token URL: The ⁠token_endpoint URL you copied.
  • Scope (optional): You can use openid as basic example. There are more pre-defined clicking on Client scopes in the left menu. The RFC specifies that, in case you want to use more than one scope, they need to be delimited with a whitespace, for example: openid email

To reduce token lifespan go to Realm settings, in the Tokens tab, update the OAuth 2.0 Device Code Lifespan to max. 2-3 minutes and Save. After that lifespan, the token will be invalid and Kibana will refresh it and call the webhook with a new one.

Once everything is configured, get a webhook url, for example from https://webhook.site/. It provides an url to use as webhook and see the content of each new incomming request. You'll need this url as the connector url field in the create connector form.

Everything is working as expected if the auth token gets updated every time it expires.

Release note

Added OAuth2 Client Credentials authentication support to Kibana Webhook connectors

@guskovaue guskovaue self-assigned this Apr 16, 2025
@guskovaue guskovaue added backport:version Backport to applied version labels v9.1.0 v8.19.0 release_note:feature Makes this part of the condensed release notes Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// labels Apr 23, 2025
@guskovaue guskovaue added release_note:enhancement and removed release_note:feature Makes this part of the condensed release notes labels Apr 23, 2025
@elastic elastic deleted a comment from elasticmachine Apr 23, 2025
@guskovaue guskovaue force-pushed the OAuth2-for-webhook-connector branch from 754b74b to 1e564ae Compare April 26, 2025 17:59
@adcoelho
Copy link
Copy Markdown
Contributor

adcoelho commented Sep 5, 2025

Why do we define additional fields instead of just scope? The documentation only seems to mention scope as a possible optional field for this grant type.

https://datatracker.ietf.org/doc/html/rfc6749#section-4.4.2

Julia added this field because https://auth0.com/, for example, needed it.

Is it the organization and audience fields?

https://auth0.com/docs/api/authentication/client-credential-flow/get-token

@jcger
Copy link
Copy Markdown
Contributor

jcger commented Sep 8, 2025

Is it the organization and audience fields?

https://auth0.com/docs/api/authentication/client-credential-flow/get-token

@adcoelho unfortunately, Julia's recordings were deleted, but the docs you shared indicate that the audience field is required

@cnasikas
Copy link
Copy Markdown
Member

cnasikas commented Sep 8, 2025

@jcger @adcoelho A lot of services require more fields when requesting a token that are specific to each service. Because they are not part of the RFC, we cannot have common fields across all services. For that reason, we introduced the "Additional fields" field so that users can put any required field they need.

@cnasikas
Copy link
Copy Markdown
Member

cnasikas commented Sep 8, 2025

@elasticmachine merge upstream

inferenceConnectorOff: false,
crowdstrikeConnectorRTROn: true,
microsoftDefenderEndpointOn: true,
isWebhookAuth2ClientCredentialsOn: false,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

used for intermediate release. Once this PR is deployed to prod, there will be a follow up PR to enable this

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will remove the feature flag because we don't need users to configure this value. Instead I'll just hardcode the value where this feature flag was used

Copy link
Copy Markdown
Member

@cnasikas cnasikas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I tested using ServiceNow and it is working as expected. I verified that a) I got an access token b) the OAuth 2.0 token was used to authenticate and create an incident in ServiceNow c) the token got refreshed successfully when it expired.

),
'data-test-subj': 'authSSL',
},
(isOAuth2Enabled || authType === AuthType.OAuth2ClientCredentials) && {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be isOAuth2Enabled && authType === AuthType.OAuth2ClientCredentials? I assume it is for the intermediate release.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because if we rollback, isOAuthEnabled will be false. This translates into: if a user created a connector using oauth2, they won't be able to edit it after the rollback

expect(result).to.eql({
status: 'ok',
connector_id: webhookActionId,
data: 'header_as_payload',
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the code in the simulator, I thought that this would contain the headers of the request. What do I miss?

Copy link
Copy Markdown
Contributor

@jcger jcger Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not in the inmediate response, the simulator stores the data and to access it you need to fetch the simulator in another request as done in line 431 const webhookSimulatorHeadersRaw = await fetch(webhookSimulatorURL). This call will respond with an array of all the payloads as you can see here

I changed the simulator so that it stores the headers in the payload array when the body is header_as_payload as you can see here - takes a while to load

@kibanamachine
Copy link
Copy Markdown
Contributor

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#9320

[✅] x-pack/platform/test/alerting_api_integration/security_and_spaces/group2/config.ts: 200/200 tests passed.

see run history

@jcger jcger requested a review from a team as a code owner September 16, 2025 10:00
</EuiFlexGroup>
<EuiSpacer size="m" />
<AuthConfig readOnly={readOnly} isPfxEnabled={isPfxEnabled} />
<React.Suspense fallback={<EuiLoadingSpinner size="m" />}>
Copy link
Copy Markdown
Contributor

@jcger jcger Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lazy loading the auth config. I did this initially because of bundle size, but I'm keeping it because it's fast even with slow 4G

Copy link
Copy Markdown
Contributor

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

packages/kbn-optimizer/limits.yml LGTM

Copy link
Copy Markdown
Member

@cnasikas cnasikas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGMT! Let's not forget to remove the isWebhookAuth2ClientCredentialsOn and rely only on if the OAuth data (secrets or config) are set.

@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
stackConnectors 362 364 +2

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
actions 347 348 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
stackConnectors 664.7KB 705.7KB +40.9KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
actions 36 37 +1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
stackConnectors 66.3KB 66.3KB -75.0B
triggersActionsUi 104.6KB 104.6KB +55.0B
total -20.0B
Unknown metric groups

API count

id before after diff
actions 353 354 +1

async chunk count

id before after diff
stackConnectors 110 109 -1

ESLint disabled line counts

id before after diff
stackConnectors 147 148 +1

Total ESLint disabled count

id before after diff
stackConnectors 151 152 +1

History

cc @jcger @guskovaue

@jcger jcger merged commit 7e64538 into elastic:main Sep 19, 2025
10 checks passed
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Sep 24, 2025
## Summary

Closes elastic#216140

We added an extra field to the OAuth 2 form called `additionalFields`.
Even though the RFC doesn't define required parameters other than the
`client id` and the `client secret`, there might be OAuth providers that
require more information.

## QA
<details>
  <summary>Token service configuration</summary>

We'll use **Keycloak** as our token service. From Wikipedia:
```
Keycloak is an open-source software product to allow single sign-on with identity and access management aimed at modern applications and services. 
...
Keycloak supports various protocols such as OpenID, OAuth version 2.0 and SAML and provides features such as user management, two-factor authentication, permissions and roles management, creating token services, etc.
```

**Step 1: Run Keycloak using Docker**

Open your terminal and run the following command:

```
docker run -p 8080:8080 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=admin quay.io/keycloak/keycloak:latest start-dev
```

**Step 2: Configure Keycloak**

1. **Access the Admin Console**:

- Open your web browser and go to ⁠http://localhost:8080.
- Click on the **Administration Console** link.
- Log in with the credentials you set in the Docker command:
	- **Username**: ⁠admin
	- **Password**: ⁠admin

2. **Create a New Realm**:
- The ⁠master realm is for managing Keycloak itself. You should create a
new one for your applications.
- In the left menu, click on **Manage realms** > **Create Realm**.
- **Realm name**: ⁠kibana-test
- Click **Create**.

3. **Create a Client**:
- A "Client" in Keycloak is an entity that can request authentication,
in our case, the Connector.
- Ensure you are in your ⁠kibana-test realm.
- In the left menu, click on **Clients** > **Create client**.
- **Client ID**: ⁠kibana-webhook-connector.
- Click **Next**.

4. **Configure Client Authentication**:
- On the next screen, enable the **Client authentication** toggle.
- Leave **Standard flow** on and enable **Service accounts role** (this
is client credentials mode)
- Click **Next** & **Save**

5. **Get Your Credentials and Token URL**:
- After saving, you'll be on the client's configuration page.
- Go to the **Credentials** tab.
- You will see the **Client secret**. Copy this value.
- Now, you need the Token URL. Go to the **Realm settings** in the left
menu.
- On the **General** tab, click the **OpenID Endpoint Configuration**
link.
- Copy the **token_endpoint**. It should be something like
`http://localhost:8080/realms/kibana-test/protocol/openid-connect/token`
Make sure the right realm name is in it, like **kibana-test** and not
`master` (yes, it happened...)

You now have the information needed:

- **Client ID**: ⁠kibana-webhook-connector
- **Client Secret**: The secret you copied from the Credentials tab.
- **Token URL**: The ⁠token_endpoint URL you copied.
- **Scope** (optional): You can use `openid` as basic example. There are
more pre-defined clicking on **Client scopes** in the left menu. The RFC
specifies that, in case you want to use more than one scope, they need
to be delimited with a whitespace, for example: `openid email`

To reduce token lifespan go to **Realm settings**, in the **Tokens**
tab, update the **OAuth 2.0 Device Code Lifespan** to max. 2-3 minutes
and **Save**. After that lifespan, the token will be invalid and Kibana
will refresh it and call the webhook with a new one.
</details>

Once everything is configured, get a webhook url, for example from
https://webhook.site/. It provides an url to use as webhook and see the
content of each new incomming request. You'll need this url as the
`connector url` field in the create connector form.

Everything is working as expected if the auth token gets updated every
time it expires.

## Release note
Added OAuth2 Client Credentials authentication support to Kibana Webhook
connectors

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christos Nasikas <christos.nasikas@elastic.co>
Co-authored-by: Julian Gernun <17549662+jcger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Christos Nasikas <xristosnasikas@gmail.com>
niros1 pushed a commit that referenced this pull request Sep 30, 2025
## Summary

Closes #216140

We added an extra field to the OAuth 2 form called `additionalFields`.
Even though the RFC doesn't define required parameters other than the
`client id` and the `client secret`, there might be OAuth providers that
require more information.

## QA
<details>
  <summary>Token service configuration</summary>

We'll use **Keycloak** as our token service. From Wikipedia:
```
Keycloak is an open-source software product to allow single sign-on with identity and access management aimed at modern applications and services. 
...
Keycloak supports various protocols such as OpenID, OAuth version 2.0 and SAML and provides features such as user management, two-factor authentication, permissions and roles management, creating token services, etc.
```

**Step 1: Run Keycloak using Docker**

Open your terminal and run the following command:

```
docker run -p 8080:8080 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=admin quay.io/keycloak/keycloak:latest start-dev
```

**Step 2: Configure Keycloak**

1. **Access the Admin Console**:

- Open your web browser and go to ⁠http://localhost:8080.
- Click on the **Administration Console** link.
- Log in with the credentials you set in the Docker command:
	- **Username**: ⁠admin
	- **Password**: ⁠admin

2. **Create a New Realm**:
- The ⁠master realm is for managing Keycloak itself. You should create a
new one for your applications.
- In the left menu, click on **Manage realms** > **Create Realm**.
- **Realm name**: ⁠kibana-test
- Click **Create**.

3. **Create a Client**:
- A "Client" in Keycloak is an entity that can request authentication,
in our case, the Connector.
- Ensure you are in your ⁠kibana-test realm.
- In the left menu, click on **Clients** > **Create client**.
- **Client ID**: ⁠kibana-webhook-connector.
- Click **Next**.

4. **Configure Client Authentication**:
- On the next screen, enable the **Client authentication** toggle.
- Leave **Standard flow** on and enable **Service accounts role** (this
is client credentials mode)
- Click **Next** & **Save**

5. **Get Your Credentials and Token URL**:
- After saving, you'll be on the client's configuration page.
- Go to the **Credentials** tab.
- You will see the **Client secret**. Copy this value.
- Now, you need the Token URL. Go to the **Realm settings** in the left
menu.
- On the **General** tab, click the **OpenID Endpoint Configuration**
link.
- Copy the **token_endpoint**. It should be something like
`http://localhost:8080/realms/kibana-test/protocol/openid-connect/token`
Make sure the right realm name is in it, like **kibana-test** and not
`master` (yes, it happened...)

You now have the information needed:

- **Client ID**: ⁠kibana-webhook-connector
- **Client Secret**: The secret you copied from the Credentials tab.
- **Token URL**: The ⁠token_endpoint URL you copied.
- **Scope** (optional): You can use `openid` as basic example. There are
more pre-defined clicking on **Client scopes** in the left menu. The RFC
specifies that, in case you want to use more than one scope, they need
to be delimited with a whitespace, for example: `openid email`

To reduce token lifespan go to **Realm settings**, in the **Tokens**
tab, update the **OAuth 2.0 Device Code Lifespan** to max. 2-3 minutes
and **Save**. After that lifespan, the token will be invalid and Kibana
will refresh it and call the webhook with a new one.
</details>

Once everything is configured, get a webhook url, for example from
https://webhook.site/. It provides an url to use as webhook and see the
content of each new incomming request. You'll need this url as the
`connector url` field in the create connector form.

Everything is working as expected if the auth token gets updated every
time it expires.

## Release note
Added OAuth2 Client Credentials authentication support to Kibana Webhook
connectors

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christos Nasikas <christos.nasikas@elastic.co>
Co-authored-by: Julian Gernun <17549662+jcger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Christos Nasikas <xristosnasikas@gmail.com>
rylnd pushed a commit to rylnd/kibana that referenced this pull request Oct 17, 2025
## Summary

Closes elastic#216140

We added an extra field to the OAuth 2 form called `additionalFields`.
Even though the RFC doesn't define required parameters other than the
`client id` and the `client secret`, there might be OAuth providers that
require more information.

## QA
<details>
  <summary>Token service configuration</summary>

We'll use **Keycloak** as our token service. From Wikipedia:
```
Keycloak is an open-source software product to allow single sign-on with identity and access management aimed at modern applications and services. 
...
Keycloak supports various protocols such as OpenID, OAuth version 2.0 and SAML and provides features such as user management, two-factor authentication, permissions and roles management, creating token services, etc.
```

**Step 1: Run Keycloak using Docker**

Open your terminal and run the following command:

```
docker run -p 8080:8080 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=admin quay.io/keycloak/keycloak:latest start-dev
```

**Step 2: Configure Keycloak**

1. **Access the Admin Console**:

- Open your web browser and go to ⁠http://localhost:8080.
- Click on the **Administration Console** link.
- Log in with the credentials you set in the Docker command:
	- **Username**: ⁠admin
	- **Password**: ⁠admin

2. **Create a New Realm**:
- The ⁠master realm is for managing Keycloak itself. You should create a
new one for your applications.
- In the left menu, click on **Manage realms** > **Create Realm**.
- **Realm name**: ⁠kibana-test
- Click **Create**.

3. **Create a Client**:
- A "Client" in Keycloak is an entity that can request authentication,
in our case, the Connector.
- Ensure you are in your ⁠kibana-test realm.
- In the left menu, click on **Clients** > **Create client**.
- **Client ID**: ⁠kibana-webhook-connector.
- Click **Next**.

4. **Configure Client Authentication**:
- On the next screen, enable the **Client authentication** toggle.
- Leave **Standard flow** on and enable **Service accounts role** (this
is client credentials mode)
- Click **Next** & **Save**

5. **Get Your Credentials and Token URL**:
- After saving, you'll be on the client's configuration page.
- Go to the **Credentials** tab.
- You will see the **Client secret**. Copy this value.
- Now, you need the Token URL. Go to the **Realm settings** in the left
menu.
- On the **General** tab, click the **OpenID Endpoint Configuration**
link.
- Copy the **token_endpoint**. It should be something like
`http://localhost:8080/realms/kibana-test/protocol/openid-connect/token`
Make sure the right realm name is in it, like **kibana-test** and not
`master` (yes, it happened...)

You now have the information needed:

- **Client ID**: ⁠kibana-webhook-connector
- **Client Secret**: The secret you copied from the Credentials tab.
- **Token URL**: The ⁠token_endpoint URL you copied.
- **Scope** (optional): You can use `openid` as basic example. There are
more pre-defined clicking on **Client scopes** in the left menu. The RFC
specifies that, in case you want to use more than one scope, they need
to be delimited with a whitespace, for example: `openid email`

To reduce token lifespan go to **Realm settings**, in the **Tokens**
tab, update the **OAuth 2.0 Device Code Lifespan** to max. 2-3 minutes
and **Save**. After that lifespan, the token will be invalid and Kibana
will refresh it and call the webhook with a new one.
</details>

Once everything is configured, get a webhook url, for example from
https://webhook.site/. It provides an url to use as webhook and see the
content of each new incomming request. You'll need this url as the
`connector url` field in the create connector form.

Everything is working as expected if the auth token gets updated every
time it expires.

## Release note
Added OAuth2 Client Credentials authentication support to Kibana Webhook
connectors

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christos Nasikas <christos.nasikas@elastic.co>
Co-authored-by: Julian Gernun <17549662+jcger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Christos Nasikas <xristosnasikas@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting release_note:feature Makes this part of the condensed release notes Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// v9.2.0

8 participants