Variables Reference ​

For a complete list of the automatically injected run metadata, see Special Environment Variables.

Environment Variables ​

System Environment Variable Filtering ​

For security, Dagu filters which system environment variables are passed to step processes and sub DAGs.

How It Works:

System environment variables are available for expansion (${VAR}) when the DAG configuration is parsed, but only filtered variables are passed to the actual step execution environment.

Filtered Variables:

Only these system environment variables are automatically passed to step processes and sub DAGs:

• Whitelisted: PATH, HOME, LANG, TZ, SHELL
• Allowed Prefixes: DAGU_*, LC_*, DAG_*

The DAG_* prefix includes the special environment variables that Dagu automatically sets (see below).

What This Means:

You can use ${AWS_SECRET_ACCESS_KEY} in your DAG YAML for variable expansion, but the AWS_SECRET_ACCESS_KEY variable itself won't be available in the environment when your step commands run unless you explicitly define it in the env section.

Defining Environment Variables ​

Set environment variables available to all steps:

env:
  - LOG_LEVEL: debug
  - DATA_DIR: /tmp/data
  - API_URL: https://api.example.com
  - API_KEY: ${SECRET_API_KEY}  # Explicitly reference system environment

Important: To use sensitive system environment variables in your workflows, you must explicitly reference them in your env section as shown above. They will not be automatically available.

Variable Expansion ​

Reference other variables:

env:
  - BASE_DIR: ${HOME}/data
  - INPUT_DIR: ${BASE_DIR}/input
  - OUTPUT_DIR: ${BASE_DIR}/output
  - CONFIG_FILE: ${INPUT_DIR}/config.yaml

Loading from .env Files ​

Load variables from dotenv files:

# Single file
dotenv: .env

# Multiple files (loaded in order)
dotenv:
  - .env
  - .env.local
  - configs/.env.${ENVIRONMENT}

Example .env file:

DATABASE_URL=postgres://localhost/mydb
API_KEY=secret123
DEBUG=true

Parameters ​

Positional Parameters ​

Define default positional parameters:

params: first second third

steps:
  - command: echo "Args: $1 $2 $3"

Run with custom values:

dagu start workflow.yaml -- one two three

Named Parameters ​

Define named parameters with defaults:

params:
  - ENVIRONMENT: dev
  - PORT: 8080
  - DEBUG: false

steps:
  - command: ./server --env=${ENVIRONMENT} --port=${PORT} --debug=${DEBUG}

Override at runtime:

dagu start workflow.yaml -- ENVIRONMENT=prod PORT=80 DEBUG=true

Accessing Parameters as JSON ​

Every step receives the full parameter map encoded as JSON via the DAGU_PARAMS_JSON environment variable. This value reflects the merged defaults plus any runtime overrides, and when a run is started with JSON parameters, the original payload is preserved.

steps:
  - name: print params
    command: echo "Raw payload: ${DAGU_PARAMS_JSON}"
  - name: batch size
    executor:
      type: jq
      config:
        raw: true
    script: ${DAGU_PARAMS_JSON}
    command: '"Batch size: \(.batchSize // "n/a")"'

Use this when downstream scripts prefer structured data or when you need access to parameters that were provided as nested JSON.

Mixed Parameters ​

Combine positional and named parameters:

params:
  - ENVIRONMENT: dev
  - VERSION: latest

steps:
  - command: echo "Deploying $1 to ${ENVIRONMENT} version ${VERSION}"

Run with:

dagu start workflow.yaml -- myapp ENVIRONMENT=prod VERSION=1.2.3

Command Substitution ​

Execute commands and use their output:

env:
  - TODAY: "`date +%Y-%m-%d`"
  - HOSTNAME: "`hostname -f`"
  - GIT_COMMIT: "`git rev-parse HEAD`"

params:
  - TIMESTAMP: "`date +%s`"
  - USER_COUNT: "`wc -l < users.txt`"

steps:
  - command: echo "Deploy on ${TODAY} from ${HOSTNAME}"

Output Variables ​

Capturing Output ​

Capture command output to use in later steps:

steps:
  - command: cat VERSION
    output: VERSION
  - command: docker build -t myapp:${VERSION} .

Output Size Limits ​

Control maximum output size:

# Global limit for all steps
maxOutputSize: 5242880  # 5MB

steps:
  - command: cat large-file.json
    output: FILE_CONTENT  # Fails if > 5MB

Redirecting Output ​

Redirect to files instead of capturing:

steps:
  - command: python report.py
    stdout: /tmp/report.txt
    stderr: /tmp/errors.log

JSON Path References ​

Access nested values in JSON output:

steps:
  - command: |
      echo '{"db": {"host": "localhost", "port": 5432}}'
    output: CONFIG
    
  - command: psql -h ${CONFIG.db.host} -p ${CONFIG.db.port}

Sub-workflow Output ​

Access outputs from nested workflows:

steps:
  - call: etl-workflow
    params: "DATE=${TODAY}"
    output: ETL_RESULT
    
  - command: |
      echo "Records processed: ${ETL_RESULT.outputs.record_count}"
      echo "Status: ${ETL_RESULT.outputs.status}"

Step ID References ​

Reference step properties using IDs:

steps:
  - id: risky
    command: 'sh -c "if [ $((RANDOM % 2)) -eq 0 ]; then echo Success; else echo Failed && exit 1; fi"'
    continueOn: failed

  - command: |
      if [ "${risky.exitCode}" = "0" ]; then
        echo "Success! Output was:"
        cat ${risky.stdout}
      else
        echo "Failed with code ${risky.exitCode}"
        cat ${risky.stderr}
      fi

Available properties:

• ${id.exitCode} - Exit code of the step
• ${id.stdout} - Path to stdout log file
• ${id.stderr} - Path to stderr log file

Variable Precedence ​

Dagu resolves variables in two places: when interpolating ${VAR} in DAG fields, and when constructing the step process environment.

Interpolation precedence (highest to lowest) ​

1. Step-level environment

   yaml

   steps:
     - env:
         - VAR: step-value

   1
   2
   3
   

2. Output variables from earlier steps

   yaml

   steps:
     - command: echo 42
       output: VAR

   1
   2
   3
   

3. Secrets

   yaml

   secrets:
     - name: VAR
       provider: env
       key: VAR

   1
   2
   3
   4
   

4. DAG-level environment (including dotenv)

   yaml

   env:
     - VAR: env-value
   dotenv: .env

   1
   2
   3
   

5. Parameters (defaults + CLI overrides)

   yaml

   params:
     - VAR: dag-default

   1
   2
   

6. Process environment fallback System environment variables are used only if the key is not set by any of the sources above, and are still subject to filtering at execution time.

Step process environment precedence (lowest to highest) ​

1. Filtered system environment
2. DAG runtime environment (params + env + .env + run metadata) If a key appears in both params and env, the env value wins.
3. Secrets
4. Step-level environment
5. Output variables from earlier steps

See Also ​

• Writing Workflows - Detailed guide on using variables
• YAML Specification - Complete YAML format reference
• Configuration Reference - Server configuration variables