How to automate infrastructure provisioning with PyInfra 3.8.0 on Linux servers

Why PyInfra 3.8.0 for Infrastructure Automation

PyInfra is a Python-based infrastructure automation tool that lets you manage servers without learning domain-specific languages like HCL or YAML configuration syntax. With the release of PyInfra 3.8.0, the tool brings performance improvements and expanded functionality for DevOps teams managing Linux infrastructure at scale.

If you're currently writing shell scripts to provision servers or managing infrastructure manually, PyInfra 3.8.0 offers a more maintainable, testable alternative that integrates seamlessly into your Python development workflow.

Key Improvements in PyInfra 3.8.0

The 3.8.0 release focuses on stability and developer experience:

• Enhanced module compatibility: Better support for modern Linux distributions (Ubuntu 24.04, Debian 12, CentOS 9)
• Improved idempotency: Operations now better detect when they've already been applied, reducing unnecessary re-runs
• Performance optimizations: Faster fact gathering and parallel execution capabilities
• Extended package manager support: Improved handling of apt, yum, and dnf across different Linux versions

Setting Up PyInfra 3.8.0

Installation on Your Control Machine

# Install PyInfra via pip
pip install pyinfra==3.8.0

# Verify installation
pyinfra --version

You'll run PyInfra from your local machine (Linux, macOS, or Windows with WSL). It connects to target servers via SSH without requiring agents.

Basic Project Structure

Create a new PyInfra project with this structure:

my-infra/
├── deploy.py          # Main deployment script
├── inventory.py       # Server inventory and groups
└── operations/
    ├── __init__.py
    ├── base.py        # Common setup operations
    └── web.py         # Web server operations

Defining Your Server Inventory

Start by creating an inventory.py file that defines your servers:

# inventory.py
from pyinfra.api import Config

# Define server groups
servers = {
    'web': [
        'web1.example.com',
        'web2.example.com',
    ],
    'database': [
        'db1.example.com',
    ]
}

# SSH configuration (optional)
config = Config(
    sudo=True,           # Use sudo for commands
    sudo_user='deploy',  # Run as specific user
    ssh_port=2222,       # Custom SSH port
    shell_executable='/bin/bash',
)

Creating Your First Deployment

Here's a practical example: provisioning a basic web server with Nginx:

# deploy.py
from pyinfra.operations import (
    apt,
    files,
    service,
    systemd,
)

# Update system packages
apt.update()

# Install required software
apt.packages(
    name='Install web server stack',
    packages=['nginx', 'curl', 'git'],
    update=True,
)

# Copy custom Nginx config
files.put(
    name='Deploy Nginx configuration',
    src='./configs/nginx.conf',
    dest='/etc/nginx/nginx.conf',
    user='root',
    group='root',
    mode='0644',
)

# Enable and start Nginx
systemd.service(
    name='Start Nginx',
    service='nginx',
    enabled=True,
    running=True,
    restarted=True,  # Restart if config changed
)

Executing Your Deployment

Run your infrastructure code against specific server groups:

# Deploy to all web servers
pyinfra inventory.py deploy.py

# Deploy only to specific group
pyinfra @web inventory.py deploy.py

# Dry-run to see what would change
pyinfra --dry inventory.py deploy.py

# Show what would be executed without running it
pyinfra --plan inventory.py deploy.py

Advanced Features in 3.8.0

Conditional Operations

Execute operations based on server facts:

from pyinfra import host

# Only run on Ubuntu systems
if 'ubuntu' in host.fact.os:
    apt.packages(packages=['ubuntu-specific-tool'])

# Only run if package not already installed
if 'nginx' not in host.fact.deb_packages:
    apt.packages(packages=['nginx'])

Using Variables and Groups

# inventory.py
servers = {
    'web': {
        'hosts': {
            'web1.example.com': {'app_env': 'production'},
            'web2.example.com': {'app_env': 'staging'},
        }
    }
}

# deploy.py
from pyinfra import host

app_env = host.data.get('app_env', 'development')

files.put(
    name=f'Deploy config for {app_env}',
    src=f'./configs/app-{app_env}.conf',
    dest='/etc/app/config.conf',
)

Comparison: PyInfra vs Alternatives

| Feature | PyInfra 3.8.0 | Ansible | Terraform | Puppet | |---------|--------------|---------|-----------|--------| | Language | Python | YAML | HCL | Ruby/Puppet DSL | | Learning curve | Moderate | Low | Moderate | High | | Agent required | No | No | No | Yes | | Idempotent | Yes | Yes | Yes | Yes | | Infrastructure state tracking | Limited | Moderate | Full (tfstate) | Full | | Best for | Python teams, Config mgmt | Multi-tool orchestration | Cloud infrastructure | Enterprise config |

Common Pitfalls with PyInfra 3.8.0

Forgetting Idempotency

Always design operations to be safe when run multiple times:

# ✅ Good: Creates file only if it doesn't exist
files.put(
    name='Create app config',
    src='app.conf',
    dest='/etc/app.conf',
    create_remote_dir=True,
)

# ❌ Bad: Overwrites file every time
files.replace(
    name='Update app config',
    path='/etc/app.conf',
    text='new content',
)

SSH Key Authentication Issues

Ensure your control machine can reach target servers:

# Test SSH connectivity
ssh -i ~/.ssh/deploy_key deploy@web1.example.com 'echo connected'

# Add key to ssh-agent for PyInfra
ssh-add ~/.ssh/deploy_key

Managing Secrets

Never hardcode passwords. Use environment variables or secret management:

import os
from pyinfra.operations import files

db_password = os.environ['DATABASE_PASSWORD']

files.template(
    name='Deploy database config',
    src='db-config.j2',
    dest='/etc/app/database.conf',
    context={'password': db_password},
)

Running PyInfra in CI/CD Pipelines

Integrate PyInfra 3.8.0 into GitHub Actions or GitLab CI:

# .github/workflows/deploy.yml
name: Deploy Infrastructure

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - run: pip install pyinfra==3.8.0
      - run: pyinfra --dry inventory.py deploy.py
      - run: pyinfra inventory.py deploy.py
        if: github.ref == 'refs/heads/main'
        env:
          SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}

Performance Tips for Large Deployments

When managing 50+ servers, optimize PyInfra's execution:

# Use parallel execution
pyinfra --parallel 10 inventory.py deploy.py

# Limit fact gathering
config = Config(
    gather_facts=False,  # Skip if not needed
    fact_cache=True,     # Cache facts between runs
)

Conclusion

PyInfra 3.8.0 brings Python's simplicity to infrastructure automation, making it ideal for teams already invested in Python workflows. With proper SSH setup, idempotent operations, and CI/CD integration, you can automate complex server provisioning without the overhead of heavier tools like Terraform or Ansible.

Start with a single server group, validate your deployment scripts with dry-runs, and gradually expand to manage your entire infrastructure declaratively.