Add fips_compatible flag to multiple packages

[shmsr]

Proposed commit message

Add a fips_compatible flag to some packages that we don't want to allow in FedRAMP env.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

[shmsr]

sql_input also needs to be disabled but migration to package-spec v3+ is something we tried earlier but we couldn't migrate.

[shmsr]

Although we want to disable sql_input but we can't as of now because of: elastic/package-spec#901

[shmsr]

Also, tests are failing as it's saying GA packages are using unreleased version: v3.4.0 of package-spec.

Error: checking package failed: linting package failed: found 4 validation errors:
  | 1. file "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1748545681270973011/elastic/integrations/packages/mongodb/manifest.yml": package with GA version (1.20.0) is using an unreleased version of the spec (3.4.0-next) (PSR00001)
  | 2. file "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1748545681270973011/elastic/integrations/packages/mongodb/data_stream/log/elasticsearch/ingest_pipeline/pipeline-json.yml" is invalid: field processors.11.remove: if is required
  | 3. file "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1748545681270973011/elastic/integrations/packages/mongodb/data_stream/log/elasticsearch/ingest_pipeline/pipeline-json.yml" is invalid: field processors.11.remove.field: rename "message" to "event.original" processor requires remove "message" processor (JSE00001)
  | 4. file "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1748545681270973011/elastic/integrations/packages/mongodb/manifest.yml" is invalid: field (root): Additional property fips_compatible is not allowed
  |  

[ishleenk17]

From which stack version onwards are we claiming FIPS compatibility?

[shmsr]

This is a package-spec thing. For stacks, versions 8.19.0 and 9.1.0 onwards are going to be FIPS 140-2 compliant but I think the change here doesn't matter regardless of the stack.

In case we fix the modules to be compliant in future, then we can bump the stack and make this flag true.

[ishleenk17]

Although we want to disable sql_input but we can't as of now because of: elastic/package-spec#901

@shmsr Since the dependency is not on this as confirmed here, are we including this for disabling fips flag ?

[shmsr]

Although we want to disable sql_input but we can't as of now because of: elastic/package-spec#901

@shmsr Since the dependency is not on this as confirmed here, are we including this for disabling fips flag ?

As pointed by Jaime, there's no version constraint for the flag. As soon as the latest spec releases, and CI passes will make the change for sql_input.

[Copilot]

Pull Request Overview

Adds a new fips_compatible flag (set to false) to several package manifests to prevent their use in FedRAMP High environments. It also bumps each package’s format_version and version, and records the change in each package’s changelog.

• Update manifest schema to 3.4.0 and bump package versions
• Add fips_compatible: false under the elastic condition
• Insert corresponding enhancement entries in each package’s changelog.yml

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
packages/sql_input/manifest.yml	Bump format & version; add fips_compatible under elastic
packages/sql_input/changelog.yml	Add changelog entry for the new fips_compatible flag
packages/postgresql/manifest.yml	Bump format & version; add fips_compatible under elastic
packages/postgresql/changelog.yml	Add changelog entry for the new fips_compatible flag
packages/oracle/manifest.yml	Bump format & version; add fips_compatible under elastic
packages/oracle/changelog.yml	Add changelog entry for the new fips_compatible flag
packages/mysql/manifest.yml	Bump format & version; add fips_compatible under elastic
packages/mysql/changelog.yml	Add changelog entry for the new fips_compatible flag
packages/mongodb/manifest.yml	Bump format & version; add fips_compatible under elastic
packages/mongodb/changelog.yml	Add changelog entry for the new fips_compatible flag
packages/microsoft_sqlserver/manifest.yml	Bump format & version; add fips_compatible under elastic
packages/microsoft_sqlserver/changelog.yml	Add changelog entry for the new fips_compatible flag
packages/azure/manifest.yml	Bump format & version; add fips_compatible under elastic
packages/azure/changelog.yml	Add changelog entry for the new fips_compatible flag

Comments suppressed due to low confidence (3)

packages/postgresql/manifest.yml:14

• [nitpick] Consider using consistent quoting for the subscription value (e.g., change to "basic") across all manifests to maintain style uniformity.

    subscription: basic

packages/azure/manifest.yml:21

• Add or update the package’s README (or central developer docs) to include an explanation of the new fips_compatible flag and its effect in FedRAMP environments.

fips_compatible: false

packages/sql_input/manifest.yml:15

• Verify that the fips_compatible field is correctly indented under the elastic section; incorrect indentation could cause YAML parsing errors or the flag to be ignored.

fips_compatible: false

[shmsr]

@jsoriano / @mrodm I am still getting the error:

3. file "/Users/subhamsarkar/go/src/github.com/elastic/integrations/build/packages/mysql-1.27.0.zip/manifest.yml" is invalid: field (root): Additional property fips_compatible is not allowed

I thought integrations repo has been updated with the latest version of ep and hence the 3.4.0 spec, so why am I getting this error. Can you please help?

[jsoriano]

@jsoriano / @mrodm I am still getting the error:

3. file "/Users/subhamsarkar/go/src/github.com/elastic/integrations/build/packages/mysql-1.27.0.zip/manifest.yml" is invalid: field (root): Additional property fips_compatible is not allowed

I thought integrations repo has been updated with the latest version of ep and hence the 3.4.0 spec, so why am I getting this error. Can you please help?

The flag was added at the package policy level, not at the root level of the manifest. Also, it was added only for integration packages, not for input packages.

I have prepared a PR with tests that illustrate where the flag is expected, and with support for input packages: elastic/package-spec#914

[shmsr]

@jsoriano / @mrodm I am still getting the error:

3. file "/Users/subhamsarkar/go/src/github.com/elastic/integrations/build/packages/mysql-1.27.0.zip/manifest.yml" is invalid: field (root): Additional property fips_compatible is not allowed

I thought integrations repo has been updated with the latest version of ep and hence the 3.4.0 spec, so why am I getting this error. Can you please help?

The flag was added at the package policy level, not at the root level of the manifest. Also, it was added only for integration packages, not for input packages.

I have prepared a PR with tests that illustrate where the flag is expected, and with support for input packages: elastic/package-spec#914

Thanks a lot @jsoriano!

@tommyers-elastic For now I am leaving out sql_input as the flag is not there for input packages; will add it later when it's available.

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[shmsr]

Can someone from @elastic/obs-ds-hosted-services also please review? Based on the modules that have been disabled in Beats, I have disabled them in Integrations as well.

SQL input still remains as the flag is not available as of yet for input packages.

[ishleenk17]

@shmsr : Changes look good.
So looks like sql input we are not doing currently.
Also the ones that are not FIPS compatible. Are there some corresponding changes for it in beats. Or are they due to the way the beats code is fetching the metrics that makes it fips incompatible ?

[shmsr]

@shmsr : Changes look good. So looks like sql input we are not doing currently. Also the ones that are not FIPS compatible. Are there some corresponding changes for it in beats. Or are they due to the way the beats code is fetching the metrics that makes it fips incompatible ?

The modules are using code that contains FIPS non-approved cryptographic algorithms, either directly or through dependencies. While we could have kept some modules enabled since we're unlikely to execute the non-compliant code paths (as per manual review), keeping them enabled helped us identify the problematic libraries in our dependency tree (e.g., azure). Therefore, we've disabled all modules that could potentially execute non-FIPS-compliant code paths.

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: fb966a3

Failed CI Steps

• Check integrations mysql

History

• 💚 Build #27906 succeeded b11fb54
• 💔 Build #27838 failed 15e940b
• 💔 Build #27835 failed 04d0573
• 💔 Build #27834 failed cec8f10

cc @shmsr

[elastic-sonarqube]

Quality Gate failed

Failed conditions
50.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

[elastic-vault-github-plugin-prod]

Package azure - 1.28.0 containing this change is available at https://epr.elastic.co/package/azure/1.28.0/

[elastic-vault-github-plugin-prod]

Package microsoft_sqlserver - 2.13.0 containing this change is available at https://epr.elastic.co/package/microsoft_sqlserver/2.13.0/

[elastic-vault-github-plugin-prod]

Package mongodb - 1.21.0 containing this change is available at https://epr.elastic.co/package/mongodb/1.21.0/

[elastic-vault-github-plugin-prod]

Package mysql - 1.27.0 containing this change is available at https://epr.elastic.co/package/mysql/1.27.0/

[elastic-vault-github-plugin-prod]

Package oracle - 1.30.0 containing this change is available at https://epr.elastic.co/package/oracle/1.30.0/

[elastic-vault-github-plugin-prod]

Package postgresql - 1.30.0 containing this change is available at https://epr.elastic.co/package/postgresql/1.30.0/