Description
Is your feature request related to a problem? Please describe.
Currently, lambda-promtail lacks the ability to parse, transform, and filter log lines within the Lambda function itself before sending them to Loki. Users who need this functionality are forced to deploy additional agents like Promtail or Grafana Alloy as intermediaries. This architecture introduces several problems.
- Increased operational overhead: Managing and maintaining additional EC2 instances or containerized deployments solely for log processing.
- Higher infrastructure costs: Running extra compute resources for intermediate agents and incurring additional network transfer costs for logs that will eventually be dropped.
- Complexity: Adds an extra layer of configuration and potential points of failure to the logging pipeline.
Describe the solution you'd like
I propose adding a configuration mechanism to lambda-promtail that allows users to define Promtail or Grafana Alloy-style pipeline stages directly via environment variables.
Specifically, I envision a new environment variable (e.g., PIPELINE_STAGES_CONFIG) where users can provide a JSON or YAML string representing an array of pipeline stage definitions, similar to how pipeline_stages/loki.process is configured in Promtail/Alloy.
Describe alternatives you've considered
Additional context
lambda-promtail already supports Prometheus-style relabeling, demonstrating its capability to process labels. Extending this to full pipeline stage support would be a natural and highly valuable evolution. It would empower users to build more efficient and cost-effective serverless logging pipelines to Loki directly