Dynamic Inventory

What is Dynamic Inventory? Dynamic inventory allows Ansible to automatically discover and track hosts from external sources that change over time. Instead of maintaining static host lists, Ansible can query cloud providers, databases, CMDB systems, and other sources to build inventory on-the-fly.

Understanding Dynamic Inventory

Why Dynamic Inventory?

Static inventory files work well for stable environments, but modern infrastructure is dynamic:

Cloud instances spin up and down on demand
Auto-scaling groups change server counts automatically
Container environments are ephemeral by nature
IP addresses change frequently
Manual inventory updates are error-prone and time-consuming

Dynamic inventory solves these challenges by querying external sources in real-time.

Use Cases

Cloud Providers: AWS, Azure, GCP, DigitalOcean
Virtualization: VMware, OpenStack, oVirt
Container Platforms: Docker, Kubernetes
Configuration Management Databases: ServiceNow, Netbox
Directory Services: LDAP, Active Directory
Infrastructure Tools: Cobbler, Foreman, Terraform state

Inventory Plugins vs Scripts

Inventory Plugins (Recommended)

Inventory plugins are the modern, preferred method for dynamic inventory:

Native Integration: Built into Ansible core
Configuration: Uses YAML configuration files
Performance: Optimized and cached
Maintained: Part of Ansible collections with regular updates
Features: Support for filters, groups, and advanced options

Inventory Scripts (Legacy)

Executable scripts that output JSON inventory:

Backwards Compatible: Still supported for legacy systems
Custom Logic: Can implement any query logic
Language Agnostic: Written in any language
Maintenance: Requires manual updates

Best Practice: Use inventory plugins for all new implementations. Only use scripts if no plugin exists for your source or you need highly customized logic.

Using Cloud Provider Inventory

AWS EC2 Inventory Plugin

The amazon.aws.aws_ec2 plugin discovers EC2 instances automatically.

Install the AWS collection:

ansible-galaxy collection install amazon.aws

Create inventory file aws_ec2.yml:

---
plugin: amazon.aws.aws_ec2

# AWS regions to query
regions:
  - us-east-1
  - us-west-2

# Filters to limit instances
filters:
  instance-state-name: running
  tag:Environment: production

# Create groups from tags
keyed_groups:
  # Group by instance type
  - key: instance_type
    prefix: instance_type

  # Group by tags
  - key: tags.Application
    prefix: app

  # Group by availability zone
  - key: placement.availability_zone
    prefix: az

# Create groups with name pattern
groups:
  webservers: "'webserver' in tags.Role"
  databases: "'database' in tags.Role"

# Compose variables from instance attributes
compose:
  ansible_host: public_ip_address
  ansible_user: "'ec2-user'"

Use the dynamic inventory:

# List all discovered hosts
ansible-inventory -i aws_ec2.yml --list

# Run playbook with dynamic inventory
ansible-playbook -i aws_ec2.yml site.yml

# Target specific dynamic group
ansible-playbook -i aws_ec2.yml site.yml --limit webservers

Azure Inventory Plugin

The azure.azcollection.azure_rm plugin discovers Azure virtual machines.

ansible-galaxy collection install azure.azcollection

Create azure_rm.yml:

---
plugin: azure.azcollection.azure_rm

# Authentication
auth_source: auto  # Uses Azure CLI or environment variables

# Include VM power states
include_vm_resource_groups:
  - my-resource-group
  - production-*

# Filter by tags
conditional_groups:
  webservers: "tags.role == 'webserver'"
  databases: "tags.role == 'database'"

# Create groups by location
keyed_groups:
  - prefix: location
    key: location

# Set connection variables
compose:
  ansible_host: public_ipv4_addresses[0]
  ansible_user: "'azureuser'"

Google Cloud Platform (GCP)

The google.cloud.gcp_compute plugin for GCP instances:

---
plugin: google.cloud.gcp_compute

projects:
  - my-gcp-project

regions:
  - us-central1
  - us-east1

filters:
  - status = RUNNING
  - labels.environment = production

keyed_groups:
  - key: labels.application
    prefix: app

  - key: zone
    prefix: zone

compose:
  ansible_host: networkInterfaces[0].accessConfigs[0].natIP
  ansible_user: "'ubuntu'"

DigitalOcean Inventory

---
plugin: community.digitalocean.digitalocean

# API authentication
api_token: "{{ lookup('env', 'DO_API_TOKEN') }}"

# Filter droplets
filters:
  - status: active
  - tag: production

# Create groups
keyed_groups:
  - key: region.slug
    prefix: region

  - key: size.slug
    prefix: size

compose:
  ansible_host: networks.v4 | selectattr('type', 'equalto', 'public') | map(attribute='ip_address') | first
  ansible_user: "'root'"

Container and Orchestration Inventory

Docker Container Inventory

Query running Docker containers as inventory:

---
plugin: community.docker.docker_containers

# Docker connection
docker_host: unix:///var/run/docker.sock

# Filter containers
filters:
  status: running
  label: managed-by-ansible

# Create groups
keyed_groups:
  - key: labels['com.docker.compose.service']
    prefix: service

# Set connection
compose:
  ansible_connection: "'docker'"
  ansible_docker_extra_args: "'--user=root'"

Kubernetes Inventory

Discover pods and services in Kubernetes:

---
plugin: kubernetes.core.k8s

# Kubeconfig file
kubeconfig: ~/.kube/config

# Namespaces to query
namespaces:
  - default
  - production

# What to include
connections:
  - namespaces: ['production']
    api_version: v1
    kind: Pod

# Create groups
keyed_groups:
  - key: metadata.namespace
    prefix: namespace

  - key: metadata.labels['app']
    prefix: app

CMDB and Infrastructure Tools

Netbox Inventory

Query infrastructure documentation from Netbox DCIM:

---
plugin: netbox.netbox.nb_inventory

# Netbox connection
api_endpoint: https://netbox.example.com
token: "{{ lookup('env', 'NETBOX_TOKEN') }}"

# Validate SSL
validate_certs: true

# Query virtual machines and devices
config_context: true

# Group by attributes
group_by:
  - device_roles
  - platforms
  - sites
  - tags

# Compose variables
compose:
  ansible_host: primary_ip4.address | ansible.utils.ipaddr('address')
  ansible_network_os: platform.slug

ServiceNow CMDB

---
plugin: servicenow.servicenow.now

# ServiceNow instance
instance: dev12345

username: "{{ lookup('env', 'SN_USERNAME') }}"
password: "{{ lookup('env', 'SN_PASSWORD') }}"

# Table to query
table: cmdb_ci_server

# Filters
query:
  operational_status: 1  # Operational
  environment: Production

# Create groups
keyed_groups:
  - key: os
    prefix: os

  - key: location.name
    prefix: location

Terraform State

Use Terraform outputs as inventory source:

---
plugin: cloud.terraform.terraform_state

# Backend configuration
backend_type: s3
backend_config:
  bucket: terraform-state
  key: infrastructure/terraform.tfstate
  region: us-east-1

# Search for resources
search_child_modules: true

# Create groups
keyed_groups:
  - key: tags.environment
    prefix: env

  - key: tags.application
    prefix: app

Combining Multiple Inventory Sources

Inventory Directory

Use a directory containing multiple inventory sources:

inventory/
├── 01-static-hosts.yml        # Static inventory
├── 02-aws-ec2.yml            # AWS dynamic inventory
├── 03-azure-rm.yml           # Azure dynamic inventory
├── 04-docker.yml             # Docker containers
└── group_vars/
    ├── all.yml
    ├── webservers.yml
    └── databases.yml

Use the entire directory as inventory:

ansible-playbook -i inventory/ site.yml

# Ansible automatically:
# 1. Reads static inventory files
# 2. Executes inventory plugins
# 3. Merges all sources
# 4. Applies group_vars

Numbering Convention: Prefix inventory files with numbers (01-, 02-) to control the order they're processed. This ensures proper precedence when variables are defined in multiple sources.

Mixed Static and Dynamic

Combine static hosts with dynamic discovery:

# static-hosts.yml
all:
  children:
    control_nodes:
      hosts:
        ansible-controller:
          ansible_host: 10.0.1.10

    bastions:
      hosts:
        bastion-01:
          ansible_host: 54.123.45.67

# aws-ec2.yml (dynamic)
plugin: amazon.aws.aws_ec2
regions:
  - us-east-1

Creating Custom Inventory Scripts

Script Requirements

Inventory scripts must support two modes:

--list - Return all groups and hosts as JSON
--host <hostname> - Return variables for specific host

Basic Script Structure (Python)

#!/usr/bin/env python3
import json
import sys

def get_inventory():
    """Return full inventory structure."""
    return {
        'webservers': {
            'hosts': ['web1.example.com', 'web2.example.com'],
            'vars': {
                'ansible_user': 'www-data',
                'http_port': 80
            }
        },
        'databases': {
            'hosts': ['db1.example.com', 'db2.example.com'],
            'vars': {
                'ansible_user': 'postgres',
                'db_port': 5432
            }
        },
        '_meta': {
            'hostvars': {
                'web1.example.com': {
                    'ansible_host': '192.168.1.10',
                    'server_id': 1
                },
                'web2.example.com': {
                    'ansible_host': '192.168.1.11',
                    'server_id': 2
                },
                'db1.example.com': {
                    'ansible_host': '192.168.1.20',
                    'replica_role': 'primary'
                },
                'db2.example.com': {
                    'ansible_host': '192.168.1.21',
                    'replica_role': 'replica'
                }
            }
        }
    }

def get_host_vars(host):
    """Return variables for specific host."""
    inventory = get_inventory()
    return inventory.get('_meta', {}).get('hostvars', {}).get(host, {})

if __name__ == '__main__':
    if len(sys.argv) == 2 and sys.argv[1] == '--list':
        print(json.dumps(get_inventory(), indent=2))
    elif len(sys.argv) == 3 and sys.argv[1] == '--host':
        print(json.dumps(get_host_vars(sys.argv[2]), indent=2))
    else:
        print("Usage: {} --list | --host ".format(sys.argv[0]))
        sys.exit(1)

Using Custom Script

# Make script executable
chmod +x custom_inventory.py

# Test the script
./custom_inventory.py --list
./custom_inventory.py --host web1.example.com

# Use with Ansible
ansible-playbook -i custom_inventory.py site.yml

Advanced Script Example (Query Database)

#!/usr/bin/env python3
import json
import sys
import pymysql

def query_database():
    """Query CMDB database for server inventory."""
    connection = pymysql.connect(
        host='cmdb.example.com',
        user='ansible',
        password='secret',
        database='infrastructure'
    )

    inventory = {
        '_meta': {
            'hostvars': {}
        }
    }

    try:
        with connection.cursor(pymysql.cursors.DictCursor) as cursor:
            # Query servers from CMDB
            cursor.execute("""
                SELECT hostname, ip_address, server_role,
                       environment, os_type
                FROM servers
                WHERE status = 'active'
            """)

            for row in cursor.fetchall():
                hostname = row['hostname']
                role = row['server_role']

                # Add to group by role
                if role not in inventory:
                    inventory[role] = {'hosts': []}
                inventory[role]['hosts'].append(hostname)

                # Add host variables
                inventory['_meta']['hostvars'][hostname] = {
                    'ansible_host': row['ip_address'],
                    'environment': row['environment'],
                    'os_type': row['os_type']
                }

    finally:
        connection.close()

    return inventory

if __name__ == '__main__':
    if len(sys.argv) == 2 and sys.argv[1] == '--list':
        print(json.dumps(query_database(), indent=2))
    elif len(sys.argv) == 3 and sys.argv[1] == '--host':
        # Return empty dict; all vars in _meta.hostvars
        print(json.dumps({}))
    else:
        sys.exit(1)

Creating Custom Inventory Plugins

Plugin Structure

Create a custom inventory plugin in plugins/inventory/custom.py:

from ansible.plugins.inventory import BaseInventoryPlugin, Constructable

DOCUMENTATION = '''
name: custom
plugin_type: inventory
short_description: Custom inventory source
description:
    - Queries custom API for inventory
options:
    plugin:
        description: Plugin name
        required: true
        choices: ['custom']
    api_url:
        description: API endpoint URL
        required: true
    api_token:
        description: Authentication token
        required: false
'''

class InventoryModule(BaseInventoryPlugin, Constructable):
    NAME = 'custom'

    def verify_file(self, path):
        """Verify this is a valid file for this plugin."""
        return path.endswith(('custom.yml', 'custom.yaml'))

    def parse(self, inventory, loader, path, cache=True):
        """Parse inventory source."""
        super(InventoryModule, self).parse(inventory, loader, path)

        # Read configuration
        self._read_config_data(path)
        api_url = self.get_option('api_url')

        # Query your API
        hosts_data = self._query_api(api_url)

        # Build inventory
        for host_data in hosts_data:
            hostname = host_data['name']

            # Add host
            self.inventory.add_host(hostname)

            # Add to groups
            for group in host_data.get('groups', []):
                self.inventory.add_group(group)
                self.inventory.add_child(group, hostname)

            # Set variables
            for key, value in host_data.get('vars', {}).items():
                self.inventory.set_variable(hostname, key, value)

    def _query_api(self, url):
        """Query API for inventory data."""
        # Implement your API query logic
        return []

Inventory Filtering and Limiting

Limit Execution to Hosts

# Run on specific host
ansible-playbook -i inventory/ site.yml --limit web1.example.com

# Run on specific group
ansible-playbook -i inventory/ site.yml --limit webservers

# Run on multiple groups
ansible-playbook -i inventory/ site.yml --limit 'webservers:databases'

# Exclude hosts
ansible-playbook -i inventory/ site.yml --limit 'all:!databases'

# Pattern matching
ansible-playbook -i inventory/ site.yml --limit 'web*.example.com'

Filter in Inventory Plugin

---
plugin: amazon.aws.aws_ec2

regions:
  - us-east-1

# Only include running instances with production tag
filters:
  instance-state-name: running
  "tag:Environment": production

# Exclude specific instances
exclude_filters:
  "tag:Exclude": "true"

# Additional filtering with compose
compose:
  skip_host: tag_Maintenance is defined and tag_Maintenance == 'true'

Performance and Caching

Enable Inventory Caching

Cache dynamic inventory to improve performance:

# ansible.cfg
[inventory]
cache = yes
cache_plugin = jsonfile
cache_timeout = 3600
cache_connection = /tmp/ansible_inventory_cache

Clear Inventory Cache

# Clear cache directory
rm -rf /tmp/ansible_inventory_cache

# Force refresh
ansible-inventory -i aws_ec2.yml --list --refresh

Per-Plugin Caching

---
plugin: amazon.aws.aws_ec2

# Enable cache for this plugin
cache: yes
cache_plugin: jsonfile
cache_timeout: 1800
cache_connection: /tmp/aws_inventory_cache

regions:
  - us-east-1

Best Practices

Dynamic Inventory Best Practices:

Use Plugins: Prefer inventory plugins over scripts for maintainability
Enable Caching: Cache inventory for large infrastructures
Filter Aggressively: Limit inventory to needed hosts at the source
Organize Groups: Create logical groups with keyed_groups
Tag Consistently: Use consistent tagging in cloud providers
Secure Credentials: Use environment variables or credential managers
Test Inventory: Use ansible-inventory --list to verify output
Document Sources: Comment inventory files with source descriptions

Common Issues and Solutions

Plugin Not Found

If inventory plugin isn't recognized:

Verify collection is installed
Check plugin name is correct
Ensure YAML file extension (.yml or .yaml)
Verify plugin: key is specified

No Hosts Returned

If inventory is empty:

Check filters aren't too restrictive
Verify authentication credentials
Test API connectivity manually
Review plugin configuration
Use ansible-inventory --list -vvv for debugging

Slow Inventory Loading

If inventory takes too long:

Enable caching
Reduce regions/zones queried
Apply filters at the source
Use multiple smaller inventory files
Consider inventory snapshots for large environments

Authentication Errors

If authentication fails:

Verify credentials are valid
Check environment variables are set
Ensure proper IAM permissions (cloud providers)
Test credentials with provider CLI tools
Review token expiration

Next Steps

Inventory Basics - Learn about static inventory
Cloud Automation - Cloud provider specific automation
Ansible Vault - Secure credentials for dynamic inventory
Playbooks - Use dynamic inventory in playbooks
Try the Playground - Practice inventory concepts

Try in Playground Practice in Labs

Getting Started

Content Distribution

Core Concepts

Platforms

Development

Infrastructure

Operations

Security

Enterprise