Skip to content
24 changes: 16 additions & 8 deletions docs/sources/shared/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -840,27 +840,35 @@ kafka_config:

The `alibabacloud_storage_config` block configures the connection to Alibaba Cloud Storage object storage backend. The supported CLI flags `<prefix>` used to reference this configuration block are:

- `common`
- `ruler`
- `common.storage`
- `ruler.storage`

&nbsp;

```yaml
# Name of OSS bucket.
# CLI flag: -<prefix>.storage.oss.bucketname
# CLI flag: -<prefix>.oss.bucketname
[bucket: <string> | default = ""]

# oss Endpoint to connect to.
# CLI flag: -<prefix>.storage.oss.endpoint
# CLI flag: -<prefix>.oss.endpoint
[endpoint: <string> | default = ""]

# alibabacloud Access Key ID
# CLI flag: -<prefix>.storage.oss.access-key-id
# CLI flag: -<prefix>.oss.access-key-id
[access_key_id: <string> | default = ""]

# alibabacloud Secret Access Key
# CLI flag: -<prefix>.storage.oss.secret-access-key
# CLI flag: -<prefix>.oss.secret-access-key
[secret_access_key: <string> | default = ""]

# Connection timeout in seconds
# CLI flag: -<prefix>.oss.conn-timeout-sec
[conn_timeout_sec: <int> | default = 5]

# Read/Write timeout in seconds
# CLI flag: -<prefix>.oss.read-write-timeout-sec
[read_write_timeout_sec: <int> | default = 5]
```

### analytics
Expand Down Expand Up @@ -1642,6 +1650,7 @@ storage:

# The alibabacloud_storage_config block configures the connection to Alibaba
# Cloud Storage object storage backend.
# The CLI flags prefix for this block configuration is: common.storage
[alibabacloud: <alibabacloud_storage_config>]

# The bos_storage_config block configures the connection to Baidu Object
Expand Down Expand Up @@ -4542,7 +4551,7 @@ storage:
[azure: <azure_storage_config>]

# Configures backend rule storage for AlibabaCloud Object Storage (OSS).
# The CLI flags prefix for this block configuration is: ruler
# The CLI flags prefix for this block configuration is: ruler.storage
[alibabacloud: <alibabacloud_storage_config>]

# Configures backend rule storage for GCS.
Expand Down Expand Up @@ -5349,7 +5358,6 @@ The `storage_config` block configures one of many possible stores for both the i
```yaml
# The alibabacloud_storage_config block configures the connection to Alibaba
# Cloud Storage object storage backend.
# The CLI flags prefix for this block configuration is: common
[alibabacloud: <alibabacloud_storage_config>]

# The aws_storage_config block configures the connection to dynamoDB and S3
Expand Down
18 changes: 13 additions & 5 deletions pkg/storage/chunk/client/alibaba/oss_object_client.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,12 @@ type OssObjectClient struct {

// OssConfig is config for the OSS Chunk Client.
type OssConfig struct {
Bucket string `yaml:"bucket"`
Endpoint string `yaml:"endpoint"`
AccessKeyID string `yaml:"access_key_id"`
SecretAccessKey string `yaml:"secret_access_key"`
Bucket string `yaml:"bucket"`
Endpoint string `yaml:"endpoint"`
AccessKeyID string `yaml:"access_key_id"`
SecretAccessKey string `yaml:"secret_access_key"`
ConnectionTimeoutSec int64 `yaml:"conn_timeout_sec"`
ReadWriteTimeoutSec int64 `yaml:"read_write_timeout_sec"`
}

// RegisterFlags registers flags.
Expand All @@ -52,11 +54,17 @@ func (cfg *OssConfig) RegisterFlagsWithPrefix(prefix string, f *flag.FlagSet) {
f.StringVar(&cfg.Endpoint, prefix+"oss.endpoint", "", "oss Endpoint to connect to.")
f.StringVar(&cfg.AccessKeyID, prefix+"oss.access-key-id", "", "alibabacloud Access Key ID")
f.StringVar(&cfg.SecretAccessKey, prefix+"oss.secret-access-key", "", "alibabacloud Secret Access Key")
f.Int64Var(&cfg.ConnectionTimeoutSec, prefix+"oss.conn-timeout-sec", 5, "Connection timeout in seconds")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: would be nice if we could revert the defaults to the existing values. 30 and 60s

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, done!

f.Int64Var(&cfg.ReadWriteTimeoutSec, prefix+"oss.read-write-timeout-sec", 5, "Read/Write timeout in seconds")
}

// NewOssObjectClient makes a new chunk.Client that writes chunks to OSS.
func NewOssObjectClient(_ context.Context, cfg OssConfig) (client.ObjectClient, error) {
client, err := oss.New(cfg.Endpoint, cfg.AccessKeyID, cfg.SecretAccessKey)
var options []oss.ClientOption
if cfg.ConnectionTimeoutSec > 0 || cfg.ReadWriteTimeoutSec > 0 {
Copy link
Contributor

@ashwanthgoli ashwanthgoli Nov 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking at the code, setting ConnectionTimeoutSec to 0 means no timeout on connection. but setting ReadWriteTimeoutSec to 0 would fail the reads and writes immediately i think

should we pass these values as is and have a validation on ReadWriteTimeoutSec during config validation to make sure it is greater than 0?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Timeout: httpTimeOut.ConnectTimeout,

c.SetReadDeadline(time.Now().Add(c.timeout))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! move value check to a separate validate function is great!
Thanks for your fast reply!

options = append(options, oss.Timeout(cfg.ConnectionTimeoutSec, cfg.ReadWriteTimeoutSec))
}
client, err := oss.New(cfg.Endpoint, cfg.AccessKeyID, cfg.SecretAccessKey, options...)
if err != nil {
return nil, err
}
Expand Down
1 change: 1 addition & 0 deletions pkg/storage/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,7 @@ type Config struct {

// RegisterFlags adds the flags required to configure this flag set.
func (cfg *Config) RegisterFlags(f *flag.FlagSet) {
cfg.AlibabaStorageConfig.RegisterFlags(f)
cfg.AWSStorageConfig.RegisterFlags(f)
cfg.AzureStorageConfig.RegisterFlags(f)
cfg.BOSStorageConfig.RegisterFlags(f)
Expand Down
Loading