Skip to content

[Feature] Support Google Authentication for Iceberg REST Catalog #62987

@tuantran0910

Description

@tuantran0910

Search before asking

  • I had searched in the issues and found no similar issues.

Description

Add support for Google Authentication Manager (GoogleAuthManager) when connecting Doris to a Google Lakehouse Iceberg REST Catalog. Currently, the Iceberg REST catalog only supports none and oauth2 security types. This feature adds a google security type that leverages Iceberg's built-in GoogleAuthManager (from the iceberg-gcp module) to authenticate using Application Default Credentials (ADC).

New configuration properties:

  • iceberg.rest.security.type - Add google as a new option alongside existing none and oauth2
  • iceberg.rest.io-impl - Set the FileIO implementation (e.g., org.apache.iceberg.gcp.gcs.GCSFileIO)
  • iceberg.rest.google.user-project - Set the Google project to be billed for API requests. This is required!
  • iceberg.gcs.oauth2.token - Provide an OAuth2 token for GCS storage access

Example usage:

CREATE CATALOG `lakehouse` PROPERTIES (
    "type" = "iceberg",
    "iceberg.catalog.type" = "rest",
    "iceberg.rest.uri" = "https://my-lakehouse.example.com",
    "iceberg.rest.security.type" = "google",
    "iceberg.rest.io-impl" = "org.apache.iceberg.gcp.gcs.GCSFileIO",
    "iceberg.rest.google.user-project" = "my-billing-project",
    "warehouse" = "gs://my-bucket/warehouse/"
);

Use case

Users running Google Cloud Dataproc or Google Lakehouse with Iceberg REST catalogs need a way to authenticate using Google's native authentication mechanism. With this support, Doris can seamlessly connect to Google-managed Iceberg catalogs using VM credentials (Application Default Credentials) without manually managing OAuth2 tokens or credentials. Ref: https://docs.cloud.google.com/lakehouse/docs/lakehouse-iceberg-rest-catalog#configure-catalog

Related issues

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions