Skip to content

Docs: Fix documentation with correct default values and add missing options #4792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jan 12, 2022
Merged

Docs: Fix documentation with correct default values and add missing options #4792

Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
65a0ce2
update docs with correct default values and add missing options
AndreZiviani Nov 19, 2021
fe20fda
fix typo
AndreZiviani Nov 19, 2021
9f58444
fix typo
AndreZiviani Nov 22, 2021
5453a6f
add another missing option
AndreZiviani Nov 22, 2021
422b018
Update docs/sources/configuration/_index.md
AndreZiviani Nov 24, 2021
1a49d64
Update docs/sources/configuration/_index.md
AndreZiviani Nov 24, 2021
d07720d
Update docs/sources/configuration/_index.md
AndreZiviani Nov 24, 2021
6d07d1d
rephrase querier_forget_delay option
AndreZiviani Nov 24, 2021
9abd4ae
remove duplicated option
AndreZiviani Nov 24, 2021
e5cab24
rephrase max_streams_per_user option
AndreZiviani Nov 24, 2021
5dc1059
fix typo
AndreZiviani Nov 24, 2021
bf19bbf
Update docs/sources/configuration/_index.md
AndreZiviani Nov 30, 2021
439e1c2
Update docs/sources/configuration/_index.md
AndreZiviani Nov 30, 2021
9cd6a8d
Merge branch 'main' into update-docs
AndreZiviani Jan 7, 2022
a117a8a
fix wrong values
AndreZiviani Jan 7, 2022
e53994c
standardize boolean naming
AndreZiviani Jan 7, 2022
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 72 additions & 29 deletions docs/sources/configuration/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,16 @@ The `frontend` block configures the Loki query-frontend.
# CLI flag: -querier.max-outstanding-requests-per-tenant
[max_outstanding_per_tenant: <int> | default = 100]

# In the event a tenant is repeatedly sending queries that lead the querier to crash
# or be killed due to an out-of-memory error, the crashed querier will be disconnected
# from the query frontend and a new querier will be immediately assigned to the tenant’s shard.
# This invalidates the assumption that shuffle sharding can be used to reduce the
# impact on tenants. This option mitigates the impact by configuring a delay between when
# a querier disconnects because of a crash and when the crashed querier is actually removed
# from the tenant's shard.
# CLI flag: -query-frontend.querier-forget-delay
[querier_forget_delay: <duration> | default = 0s]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loki doesn't have support for querier shuffle sharding yet, I wonder if we should keep this configuration removed until it does?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a problem leaving as is because its not only related to shuffle sharding but I can remove it no problem
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@trevorwhitney any input here?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have as much context as @slim-bean, are you saying this value isn't even used by loki yet because tenants don't have shards? If so I can see the point that adding this field to the docs because changing it won't actually do anything. At the same time it is a setting we have that will appear when someone hits the /config endpoint or prints the config to stderr, so I don't think there's too much harm in documenting it to help explain how that value got there.

# Compress HTTP responses.
# CLI flag: -querier.compress-http-responses
[compress_responses: <boolean> | default = false]
Expand All @@ -359,7 +369,7 @@ The `frontend` block configures the Loki query-frontend.
# query-scheduler instances.
# Also used to determine how often to poll the scheduler-ring for addresses if configured.
# CLI flag: -frontend.scheduler-dns-lookup-period
[scheduler_dns_lookup_period: <duration> | default = 3s]
[scheduler_dns_lookup_period: <duration> | default = 10s]

# Number of concurrent workers forwarding queries to single query-scheduler.
# CLI flag: -frontend.scheduler-worker-concurrency
Expand Down Expand Up @@ -545,7 +555,7 @@ remote_write:
[proxy_url: <string>]

# Configure whether HTTP requests follow HTTP 3xx redirects.
[follow_redirects: <bool> | default = true]
[follow_redirects: <boolean> | default = true]

# Configures the queue used to write to remote storage.
queue_config:
Expand Down Expand Up @@ -719,7 +729,7 @@ The `gcs_storage_config` configures GCS as a general storage for different data

# Enable HTTP/2 when connecting to GCS.
# CLI flag: -<prefix>.gcs.enable-http2
[enable_http2: <bool> | default = true]
[enable_http2: <boolean> | default = true]
```

## s3_storage_config
Expand Down Expand Up @@ -898,6 +908,10 @@ The `frontend_worker` configures the worker - running within the Loki querier -
# CLI flag: -querier.worker-parallelism
[parallelism: <int> | default = 10]

# Force worker concurrency to match the -querier.max-concurrent option. Overrides querier.worker-parallelism.
# CLI flag: -querier.worker-match-max-concurrent
[match_max_concurrent: <boolean> | default = false]

# How often to query the frontend_address DNS to resolve frontend addresses.
# Also used to determine how often to poll the scheduler-ring for addresses if configured.
# CLI flag: -querier.dns-lookup-period
Expand Down Expand Up @@ -991,10 +1005,14 @@ lifecycler:
# CLI flag: -ingester.join-after
[join_after: <duration> | default = 0s]

# Observe tokens after generating to resolve collisions. Useful when using a gossip ring.
# CLI flag: -ingester.observe-period
[observe_period: <duration> | default = 0s]

# Minimum duration to wait before becoming ready. This is to work around race
# conditions with ingesters exiting and updating the ring.
# CLI flag: -ingester.min-ready-duration
[min_ready_duration: <duration> | default = 1m]
[min_ready_duration: <duration> | default = 15s]

# Name of network interfaces to read addresses from.
# CLI flag: -ingester.lifecycler.interface
Expand All @@ -1004,12 +1022,12 @@ lifecycler:

# Duration to sleep before exiting to ensure metrics are scraped.
# CLI flag: -ingester.final-sleep
[final_sleep: <duration> | default = 0s]
[final_sleep: <duration> | default = 30s]

# Number of times to try and transfer chunks when leaving before
# falling back to flushing to the store. Zero = no transfers are done.
# CLI flag: -ingester.max-transfer-retries
[max_transfer_retries: <int> | default = 10]
[max_transfer_retries: <int> | default = 0]

# How many flushes can happen concurrently from each stream.
# CLI flag: -ingester.concurrent-flushes
Expand All @@ -1025,7 +1043,7 @@ lifecycler:

# How long chunks should be retained in-memory after they've been flushed.
# CLI flag: -ingester.chunks-retain-period
[chunk_retain_period: <duration> | default = 15m]
[chunk_retain_period: <duration> | default = 0s]

# How long chunks should sit in-memory with no updates before
# being flushed if they don't hit the max block size. This means
Expand All @@ -1042,10 +1060,10 @@ lifecycler:
# A target _compressed_ size in bytes for chunks.
# This is a desired size not an exact size, chunks may be slightly bigger
# or significantly smaller if they get flushed for other reasons (e.g. chunk_idle_period)
# The default value of 0 for this will create chunks with a fixed 10 blocks,
# A value of 0 creates chunks with a fixed 10 blocks,
# A non zero value will create chunks with a variable number of blocks to meet the target size.
# CLI flag: -ingester.chunk-target-size
[chunk_target_size: <int> | default = 0]
[chunk_target_size: <int> | default = 1572864]

# The compression algorithm to use for chunks. (supported: gzip, lz4, snappy)
# You should choose your algorithm depending on your need:
Expand Down Expand Up @@ -1482,23 +1500,23 @@ cassandra:
# Instruct the Cassandra driver to not attempt to get host
# info from the system.peers table.
# CLI flag: -cassandra.disable-initial-host-lookup
[disable_initial_host_lookup: <bool> | default = false]
[disable_initial_host_lookup: <boolean> | default = false]

# Use SSL when connecting to Cassandra instances.
# CLI flag: -cassandra.ssl
[SSL: <boolean> | default = false]

# Require SSL certificate validation when SSL is enabled.
# CLI flag: -cassandra.host-verification
[host_verification: <bool> | default = true]
[host_verification: <boolean> | default = true]

# Path to certificate file to verify the peer when SSL is enabled.
# CLI flag: -cassandra.ca-path
[CA_path: <string>]

# Enable password authentication when connecting to Cassandra.
# CLI flag: -cassandra.auth
[auth: <bool> | default = false]
[auth: <boolean> | default = false]

# Username for password authentication when auth is true.
# CLI flag: -cassandra.username
Expand Down Expand Up @@ -1708,7 +1726,7 @@ memcached:

# Configures how many keys to fetch in each batch request.
# CLI flag: -<prefix>.memcached.batchsize
batch_size: <int>
batch_size: <int> | default = 1024

# Maximum active requests to memcached.
# CLI flag: -<prefix>.memcached.parallelism
Expand Down Expand Up @@ -1744,7 +1762,27 @@ memcached_client:

# Whether or not to use a consistent hash to discover multiple memcached servers.
# CLI flag: -<prefix>.memcached.consistent-hash
[consistent_hash: <bool>]
[consistent_hash: <boolean> | default = true]

# Trip the circuit breaker after this number of consecutive dial failures.
# A value of 0 disables the circuit breaker.
# CLI flag: -<prefix>.memcached.circuit-breaker-consecutive-failures
[circuit_breaker_consecutive_failures: <int> | default = 10]

# Duration the circuit breaker remains open after tripping.
# If set to 0, the duration is 60 seconds.
# CLI flag: -<prefix>.memcached.circuit-breaker-timeout
[circuit_breaker_timeout: <duration> | default = 10s]

# Reset the circuit breaker counts after this duration.
# A value of 0 never resets the circuit breaker.
# CLI flag: -<prefix>.memcached.circuit-breaker-interval
[circuit_breaker_interval: <duration> | default = 10s]

# The maximum size of an item stored in memcached.
# Bigger items are not stored. If set to 0, no maximum size is enforced.
# CLI flag: -<prefix>.memcached.max-item-size
[max_item_size: <int> | default = 0]

redis:
# Redis Server or Cluster configuration endpoint to use for caching. A comma-separated list of endpoints
Expand All @@ -1758,15 +1796,15 @@ redis:

# Maximum time to wait before giving up on redis requests.
# CLI flag: -<prefix>.redis.timeout
[timeout: <duration> | default = 100ms]
[timeout: <duration> | default = 500ms]

# How long keys stay in the redis.
# CLI flag: -<prefix>.redis.expiration
[expiration: <duration> | default = 0s]

# Database index.
# CLI flag: -<prefix>.redis.db
[db: <int>]
[db: <int> | default = 0]

# Maximum number of connections in the pool.
# CLI flag: -<prefix>.redis.pool-size
Expand All @@ -1780,6 +1818,10 @@ redis:
# CLI flag: -<prefix>.redis.tls-enabled
[tls_enabled: <boolean> | default = false]

# Skip validating server certificate.
# CLI flag: -<prefix>.redis.tls-insecure-skip-verify
[tls_insecure_skip_verify: <boolean> | default = false]

# Close connections after remaining idle for this duration.
# If the value is zero, then idle connections are not closed.
# CLI flag: -<prefix>.redis.idle-timeout
Expand All @@ -1801,8 +1843,9 @@ fifocache:
[max_size_items: <int> | default = 0]

# The expiry duration for the cache.
# The default value of 0 disables expiration.
# CLI flag: -<prefix>.fifocache.duration
[validity: <duration> | default = 1h]
[validity: <duration>]
```

## schema_config
Expand Down Expand Up @@ -1893,7 +1936,7 @@ compacts index shards to more performant forms.

# (Experimental) Activate custom (per-stream,per-tenant) retention.
# CLI flag: -boltdb.shipper.compactor.retention-enabled
[retention_enabled: <bool> | default = false]
[retention_enabled: <boolean> | default = false]

# Delay after which chunks will be fully deleted during retention.
# CLI flag: -boltdb.shipper.compactor.retention-delete-delay
Expand Down Expand Up @@ -1965,7 +2008,7 @@ The `limits_config` block configures global and per-tenant limits in Loki.

# Whether or not old samples will be rejected.
# CLI flag: -validation.reject-old-samples
[reject_old_samples: <bool> | default = true]
[reject_old_samples: <boolean> | default = true]

# Maximum accepted sample age before rejecting.
# CLI flag: -validation.reject-old-samples.max-age
Expand All @@ -1980,14 +2023,10 @@ The `limits_config` block configures global and per-tenant limits in Loki.
# CLI flag: -validation.enforce-metric-name
[enforce_metric_name: <boolean> | default = true]

# Maximum number of active streams per user, per ingester. 0 to disable.
# CLI flag: -ingester.max-streams-per-user
[max_streams_per_user: <int> | default = 0]

# Maximum line size on ingestion path. Example: 256kb.
# There is no limit when unset.
# There is no limit when unset or set to 0.
# CLI flag: -distributor.max-line-size
[max_line_size: <string> | default = none ]
[max_line_size: <string> | default = 0 ]

# Truncate log lines when they exceed max_line_size.
# CLI flag: -distributor.max-line-size-truncate
Expand All @@ -1997,6 +2036,10 @@ The `limits_config` block configures global and per-tenant limits in Loki.
# CLI flag: -validation.max-entries-limit
[max_entries_limit_per_query: <int> | default = 5000 ]

# Maximum number of active streams per user, per ingester. 0 to make it unlimited.
# CLI flag: -ingester.max-streams-per-user
[max_streams_per_user: <int> | default 0]

# Maximum number of active streams per user, across the cluster. 0 to disable.
# When the global limit is enabled, each ingester is configured with a dynamic
# local limit based on the replication factor and the current number of healthy
Expand All @@ -2006,7 +2049,7 @@ The `limits_config` block configures global and per-tenant limits in Loki.

# When true, out-of-order writes are accepted.
# CLI flag: -ingester.unordered-writes
[unordered_writes: <bool> | default = true]
[unordered_writes: <boolean> | default = true]

# Maximum number of chunks that can be fetched by a single query.
# CLI flag: -store.query-chunk-limit
Expand Down Expand Up @@ -2113,7 +2156,7 @@ The `limits_config` block configures global and per-tenant limits in Loki.
[max_query_lookback: <duration> | default = 0]

# Disable recording rules remote-write.
[ruler_remote_write_disabled: <bool> | default = false]
[ruler_remote_write_disabled: <boolean> | default = false]

# The URL of the endpoint to send samples to.
[ruler_remote_write_url: <string>]
Expand Down Expand Up @@ -2153,7 +2196,7 @@ The `limits_config` block configures global and per-tenant limits in Loki.
[ruler_remote_write_queue_max_backoff: <duration>]
# Retry upon receiving a 429 status code from the remote-write storage.
# This is experimental and might change in the future.
[ruler_remote_write_queue_retry_on_ratelimit: <bool>]
[ruler_remote_write_queue_retry_on_ratelimit: <boolean>]

# Limit queries that can be sharded.
# Queries within the time range of now and now minus this sharding lookback
Expand Down Expand Up @@ -2198,7 +2241,7 @@ The `grpc_client_config` block configures a client connection to a gRPC service.

# Enable backoff and retry when a rate limit is hit.
# CLI flag: -<prefix>.backoff-on-ratelimits
[backoff_on_ratelimits: <bool> | default = false]
[backoff_on_ratelimits: <boolean> | default = false]

# Configures backoff when enabled.
backoff_config:
Expand Down