Nifi-Python-Api: A rich Apache NiFi Python Client SDK

Multiple interfaces for working with Apache NiFi:

• Command-Line Interface: Shell access for scripting and CI/CD pipelines (pip install nipyapi[cli])
• Mid-level Client SDK: Python functions for typical complex tasks
• Low-level Client SDKs: Full API implementation of NiFi and NiFi Registry
• Examples: Ready-to-use scripts in the examples directory

Functionality Highlights:

• CI/CD Operations: Purpose-built commands for flow deployment pipelines (nipyapi.ci)
• Profiles System: One-command environment switching with nipyapi.profiles.switch('single-user') (see profiles documentation)
• Modern Authentication: Built-in support for Basic Auth, mTLS, OIDC/OAuth2, and LDAP
• Environment Management: YAML/JSON configuration with environment variable overrides
• CRUD wrappers for common task areas like Processor Groups, Processors, Clients, Buckets, Flows, etc.
• Convenience functions for inventory tasks, such as recursively retrieving the entire canvas, or a flat list of all Process Groups
• Support for scheduling and purging flows, controller services, and connections
• Support for import/export of Versioned Flows from various sources
• Integrated Docker workflow with Makefile automation and profile-based testing

Please see the issue register for more information on current development.

You need a running NiFi instance to connect to. Choose the approach that fits your situation:

Path A: Quick Start with Docker (Recommended for New Users)

Use our provided Docker environment for immediate testing:

# Clone the repository (includes Docker profiles and Makefile)
git clone https://github.com/Chaffelson/nipyapi.git
cd nipyapi

# Install NiPyAPI in development mode
pip install -e ".[dev]"

# Start complete NiFi environment (this may take a few minutes)
make certs && make up NIPYAPI_PROFILE=single-user && make wait-ready NIPYAPI_PROFILE=single-user

# Test the connection
python3 -c "
import nipyapi
nipyapi.profiles.switch('single-user')
version = nipyapi.system.get_nifi_version_info()
print(f'✓ Connected to NiFi {version}')
"

Path B: Connect to Your Existing NiFi

If you already have NiFi running, install and configure:

# Install NiPyAPI
pip install nipyapi

# Create your own profiles.yml
mkdir -p ~/.nipyapi
cat > ~/.nipyapi/profiles.yml << EOF
my-nifi:
  nifi_url: https://your-nifi-host.com/nifi-api
  registry_url: http://your-registry-host.com/nifi-registry-api
  nifi_user: your_username
  nifi_pass: your_password
  nifi_verify_ssl: true
EOF

# Test your custom profile
python3 -c "
import nipyapi
nipyapi.config.default_profiles_file = '~/.nipyapi/profiles.yml'
nipyapi.profiles.switch('my-nifi')
version = nipyapi.system.get_nifi_version_info()
print(f'✓ Connected to NiFi {version}')
"

Path C: Manual Configuration (Advanced)

For advanced use cases without profiles:

# Install NiPyAPI
pip install nipyapi

# Configure in Python code
import nipyapi
from nipyapi import config, utils

# Configure endpoints
config.nifi_config.host = 'https://your-nifi-host.com/nifi-api'
config.registry_config.host = 'http://your-registry-host.com/nifi-registry-api'

# Configure authentication
utils.set_endpoint(config.nifi_config.host, ssl=True, login=True,
                   username='your_username', password='your_password')

Next Steps: Start Using NiPyAPI

Once your environment is set up, you can start using NiPyAPI:

import nipyapi

# If using profiles (Paths A or B)
nipyapi.profiles.switch('single-user')  # or your custom profile name

# Basic operations
root_pg_id = nipyapi.canvas.get_root_pg_id()
version = nipyapi.system.get_nifi_version_info()
process_groups = nipyapi.canvas.list_all_process_groups()

print(f"Connected to NiFi {version}")
print(f"Root Process Group: {root_pg_id}")
print(f"Found {len(process_groups)} process groups")

Command-Line Interface:

For shell scripting, CI/CD pipelines, and agentic workflows, install the CLI extras:

pip install "nipyapi[cli]"

# Set connection via environment variables
export NIFI_API_ENDPOINT=https://your-nifi-host.com/nifi-api
export NIFI_USERNAME=your_username
export NIFI_PASSWORD=your_password

# Or use a profile
nipyapi --profile single-user system get_nifi_version_info

# CI/CD operations
nipyapi ci deploy_flow --bucket flows --flow my-flow
nipyapi ci start_flow --process_group_id <id>
nipyapi ci get_status --process_group_id <id>

See docs/cli.rst for the complete CLI reference and docs/ci.rst for CI/CD pipeline examples. See the companion repository nipyapi-actions for GitHub Actions and GitLab CI examples and support.

Built-in Docker Profiles:

When using Path A (Docker), these profiles are available:

• single-user - HTTP Basic authentication (easiest to start with)
• secure-ldap - LDAP authentication over TLS
• secure-mtls - Mutual TLS certificate authentication
• secure-oidc - OpenID Connect (OAuth2) authentication

See docs/profiles.rst for complete profiles documentation and docs/migration.rst for upgrading from 0.x.

Examples and Advanced Usage:

• Flow Development Lifecycle: See examples/fdlc.py for multi-environment workflow patterns
• Interactive Sandbox: Run make sandbox NIPYAPI_PROFILE=single-user for experimentation (requires Docker setup)
• Custom Profiles: Create your own profiles.yml for production environments
• Environment Variables: Override any setting with NIFI_API_ENDPOINT, NIFI_USERNAME, etc.
• Testing Different Auth Methods: Use make up NIPYAPI_PROFILE=secure-ldap to try LDAP authentication

Please check out the Contribution Guide if you are interested in contributing to the feature set.