Ansible Vault

What is Ansible Vault?

Ansible Vault is a feature that allows you to encrypt sensitive data such as passwords, API keys, and certificates. This keeps your secrets secure while still allowing you to version control your Ansible files.

Basic Vault Commands

Create Encrypted File

# Create a new encrypted file
ansible-vault create secrets.yml

# You'll be prompted for a password
# Then enter your secrets in YAML format
---
db_password: SuperSecretPassword123
api_key: abc123def456ghi789

Edit Encrypted File

ansible-vault edit secrets.yml

View Encrypted File

ansible-vault view secrets.yml

Encrypt Existing File

ansible-vault encrypt existing_file.yml

Decrypt File

# Decrypt to view (use sparingly!)
ansible-vault decrypt secrets.yml

# Re-encrypt after viewing
ansible-vault encrypt secrets.yml

Rekey (Change Password)

ansible-vault rekey secrets.yml

Using Vault in Playbooks

Method 1: Encrypted Variable File

Create encrypted vars file:

ansible-vault create group_vars/all/vault.yml

Content of vault.yml:

---
vault_db_password: SuperSecret123
vault_api_key: abc123xyz789
vault_ssh_private_key: |
  -----BEGIN RSA PRIVATE KEY-----
  MIIEpAIBAAKCAQEA...
  -----END RSA PRIVATE KEY-----

Reference in regular vars file (group_vars/all/vars.yml):

---
db_password: "{{ vault_db_password }}"
api_key: "{{ vault_api_key }}"

Use in playbook:

---
- name: Deploy application
  hosts: app_servers

  tasks:
    - name: Configure database connection
      template:
        src: db_config.j2
        dest: /etc/app/db.conf
      vars:
        database_password: "{{ db_password }}"

Method 2: Encrypt Specific Variables

# Encrypt a string
ansible-vault encrypt_string 'SuperSecret123' --name 'db_password'

# Output:
db_password: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          66386439653765386661616536373439...

Use directly in vars file:

---
db_user: admin
db_password: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          66386439653765386661616536373439...
db_host: localhost

Running Playbooks with Vault

Prompt for Vault Password

ansible-playbook site.yml --ask-vault-pass

Use Password File

# Create password file
echo 'MyVaultPassword' > .vault_pass
chmod 600 .vault_pass

# Add to .gitignore
echo '.vault_pass' >> .gitignore

# Use in playbook
ansible-playbook site.yml --vault-password-file .vault_pass

Configure in ansible.cfg

[defaults]
vault_password_file = .vault_pass

Now run without specifying password:

ansible-playbook site.yml

Multiple Vault Passwords

Useful for different teams or security levels.

Encrypt with Vault ID

# Create file with 'dev' vault ID
ansible-vault create --vault-id dev@prompt secrets_dev.yml

# Create file with 'prod' vault ID
ansible-vault create --vault-id prod@prompt secrets_prod.yml

Use Multiple Vault Passwords

ansible-playbook site.yml \
  --vault-id dev@.vault_pass_dev \
  --vault-id prod@.vault_pass_prod

Vault Password Scripts

Use a script to retrieve passwords from external sources.

#!/usr/bin/env python3
# vault_pass.py
import os
import sys

# Get password from environment variable
password = os.environ.get('ANSIBLE_VAULT_PASSWORD')
if password:
    print(password)
    sys.exit(0)
else:
    sys.exit(1)

chmod +x vault_pass.py
export ANSIBLE_VAULT_PASSWORD='MySecretPassword'
ansible-playbook site.yml --vault-password-file ./vault_pass.py

Best Practices

Directory Structure

ansible-project/
├── ansible.cfg
├── .vault_pass          # Never commit this!
├── .gitignore
├── inventory/
│   └── hosts
├── group_vars/
│   ├── all/
│   │   ├── vars.yml     # Non-sensitive vars
│   │   └── vault.yml    # Encrypted sensitive vars
│   └── production/
│       ├── vars.yml
│       └── vault.yml
└── playbooks/
    └── site.yml

.gitignore

.vault_pass
*.vault_key
*_vault_pass*

Complete Example

1. Create Vault File

ansible-vault create group_vars/production/vault.yml

Content:

---
vault_mysql_root_password: RootPass123!
vault_mysql_app_password: AppPass456!
vault_aws_access_key: AKIAIOSFODNN7EXAMPLE
vault_aws_secret_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

2. Create Regular Vars File

group_vars/production/vars.yml:

---
mysql_root_password: "{{ vault_mysql_root_password }}"
mysql_app_password: "{{ vault_mysql_app_password }}"
aws_access_key: "{{ vault_aws_access_key }}"
aws_secret_key: "{{ vault_aws_secret_key }}"

# Non-sensitive vars
mysql_host: localhost
mysql_port: 3306
app_name: myapp

3. Use in Playbook

---
- name: Setup MySQL database
  hosts: database_servers
  become: yes

  tasks:
    - name: Set MySQL root password
      mysql_user:
        name: root
        password: "{{ mysql_root_password }}"
        host: localhost

    - name: Create application database
      mysql_db:
        name: "{{ app_name }}"
        state: present

    - name: Create application user
      mysql_user:
        name: "{{ app_name }}_user"
        password: "{{ mysql_app_password }}"
        priv: "{{ app_name }}.*:ALL"

4. Run Playbook

ansible-playbook database.yml --ask-vault-pass

Advanced: Encrypting Files

Encrypt Binary Files

# Encrypt SSL certificate
ansible-vault encrypt ssl/server.key

# Use in playbook
- name: Copy SSL key
  copy:
    src: ssl/server.key
    dest: /etc/ssl/private/server.key
    mode: '0600'

Security Recommendations

Troubleshooting

Check if File is Encrypted

head -1 secrets.yml
# Should show: $ANSIBLE_VAULT;1.1;AES256

Decryption Failed

# Wrong password error
ERROR! Decryption failed (no vault secrets were found)

# Solution: Verify password or vault ID
ansible-vault view --vault-id @prompt secrets.yml

Debug Vault Loading

ansible-playbook site.yml --ask-vault-pass -vvv

Advanced Encryption Workflows

File-Level vs Variable-Level Encryption

File-Level Encryption: Best for files with only sensitive data

# Encrypt entire file
ansible-vault encrypt group_vars/production/vault.yml

# Pros:
# - Simple to manage
# - Clear separation of sensitive data
# - Easy to audit what's encrypted

# Cons:
# - Can't see file structure in git diff
# - Must decrypt entire file to view any value

Variable-Level Encryption: Best for mixed sensitive/non-sensitive data

# Encrypt individual variables
ansible-vault encrypt_string 'secret_value' --name 'my_secret'

# Example in vars file:
---
# Non-sensitive - visible in git
app_name: myapp
app_port: 8080
app_user: appuser

# Sensitive - encrypted inline
app_password: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          66386439653765386661616536373439...

api_key: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          33343536373839613031323334353637...

# Pros:
# - See file structure in git diff
# - Mix sensitive and non-sensitive data
# - Granular encryption control

# Cons:
# - More complex to manage
# - Harder to rotate all secrets at once

Hybrid Approach (Recommended)

# group_vars/production/vars.yml (unencrypted)
---
# All variables reference encrypted values
db_host: "{{ vault_db_host }}"
db_port: "{{ vault_db_port }}"
db_user: "{{ vault_db_user }}"
db_password: "{{ vault_db_password }}"

api_endpoint: "{{ vault_api_endpoint }}"
api_key: "{{ vault_api_key }}"

# Non-sensitive vars
app_name: myapp
app_version: 1.0.0

# group_vars/production/vault.yml (encrypted file)
---
vault_db_host: prod-db.internal.example.com
vault_db_port: 5432
vault_db_user: prod_app_user
vault_db_password: SuperSecretDBPass123!

vault_api_endpoint: https://api.internal.example.com
vault_api_key: sk_live_abcdef123456789

# Benefit: Clean separation, easy rotation, visible structure

Integration with External Secret Managers

HashiCorp Vault Integration

Use HashiCorp Vault as the central secret store for enterprise-grade secret management.

Install the hashi_vault lookup plugin:

# Install collection
ansible-galaxy collection install community.hashi_vault

# Verify installation
ansible-doc -t lookup community.hashi_vault.hashi_vault

Configure Vault authentication:

# Method 1: Token authentication
---
- name: Fetch secrets from HashiCorp Vault
  hosts: localhost
  vars:
    vault_token: "{{ lookup('env', 'VAULT_TOKEN') }}"
    vault_url: "https://vault.example.com:8200"

  tasks:
    - name: Fetch database password
      set_fact:
        db_password: "{{ lookup('community.hashi_vault.hashi_vault',
                         'secret=secret/data/myapp/database:password
                         url=' + vault_url + '
                         token=' + vault_token) }}"

    - name: Use the secret
      debug:
        msg: "Database password retrieved (length: {{ db_password | length }})"

Using AppRole authentication (recommended for automation):

---
- name: HashiCorp Vault with AppRole
  hosts: app_servers
  vars:
    vault_url: "https://vault.example.com:8200"
    vault_role_id: "{{ lookup('env', 'VAULT_ROLE_ID') }}"
    vault_secret_id: "{{ lookup('env', 'VAULT_SECRET_ID') }}"

  tasks:
    - name: Fetch application secrets
      set_fact:
        app_secrets: "{{ lookup('community.hashi_vault.hashi_vault',
                         'secret=secret/data/myapp/prod:data
                         url=' + vault_url + '
                         auth_method=approle
                         role_id=' + vault_role_id + '
                         secret_id=' + vault_secret_id) }}"

    - name: Configure application with secrets
      template:
        src: app_config.j2
        dest: /etc/myapp/config.yml
      vars:
        db_password: "{{ app_secrets.db_password }}"
        api_key: "{{ app_secrets.api_key }}"
      no_log: true

Vault KV v2 with version support:

---
- name: Fetch specific secret version
  hosts: localhost
  tasks:
    - name: Get secret from specific version
      set_fact:
        secret: "{{ lookup('community.hashi_vault.hashi_vault',
                    'secret=secret/data/myapp/config:password
                    version=3
                    url=https://vault.example.com:8200') }}"

    - name: Get all versions metadata
      uri:
        url: "https://vault.example.com:8200/v1/secret/metadata/myapp/config"
        headers:
          X-Vault-Token: "{{ vault_token }}"
      register: secret_metadata

    - debug:
        msg: "Secret has {{ secret_metadata.json.data.versions | length }} versions"

AWS Secrets Manager Integration

# Install collection
ansible-galaxy collection install amazon.aws

# Playbook using AWS Secrets Manager
---
- name: Fetch secrets from AWS Secrets Manager
  hosts: app_servers
  vars:
    aws_region: us-east-1

  tasks:
    - name: Get database credentials
      set_fact:
        db_creds: "{{ lookup('amazon.aws.aws_secret',
                     'prod/myapp/database',
                     region=aws_region) | from_json }}"

    - name: Configure database connection
      template:
        src: database.conf.j2
        dest: /etc/myapp/database.conf
        mode: '0600'
      vars:
        db_host: "{{ db_creds.host }}"
        db_user: "{{ db_creds.username }}"
        db_password: "{{ db_creds.password }}"
      no_log: true

    - name: Get API keys
      set_fact:
        api_keys: "{{ lookup('amazon.aws.aws_secret',
                     'prod/myapp/api-keys',
                     region=aws_region,
                     on_denied='error') | from_json }}"

Using AWS Parameter Store:

---
- name: Fetch from AWS Systems Manager Parameter Store
  hosts: localhost
  tasks:
    - name: Get secure string parameter
      set_fact:
        db_password: "{{ lookup('amazon.aws.aws_ssm',
                        '/myapp/prod/db_password',
                        decrypt=true,
                        region='us-east-1') }}"

    - name: Get multiple parameters by path
      set_fact:
        all_params: "{{ lookup('amazon.aws.aws_ssm',
                       '/myapp/prod/',
                       bypath=true,
                       recursive=true,
                       decrypt=true) }}"

    - name: Use parameters
      debug:
        msg: "Retrieved {{ all_params | length }} parameters"

Azure Key Vault Integration

# Install collection
ansible-galaxy collection install azure.azcollection
pip3 install 'ansible[azure]'

# Playbook using Azure Key Vault
---
- name: Fetch secrets from Azure Key Vault
  hosts: localhost
  vars:
    vault_name: myapp-prod-kv
    azure_subscription_id: "{{ lookup('env', 'AZURE_SUBSCRIPTION_ID') }}"

  tasks:
    - name: Get secret from Key Vault
      azure_rm_keyvaultsecret_info:
        vault_uri: "https://{{ vault_name }}.vault.azure.net"
        name: database-password
      register: db_secret

    - name: Use the secret
      debug:
        msg: "Secret retrieved successfully"
      when: db_secret.secrets | length > 0

    - name: Get certificate from Key Vault
      azure_rm_keyvaultcertificate_info:
        vault_uri: "https://{{ vault_name }}.vault.azure.net"
        name: ssl-certificate
      register: ssl_cert

    - name: Get multiple secrets
      azure_rm_keyvaultsecret_info:
        vault_uri: "https://{{ vault_name }}.vault.azure.net"
      register: all_secrets

    - name: Process secrets
      set_fact:
        api_key: "{{ (all_secrets.secrets | selectattr('name', 'equalto', 'api-key') | first).value }}"

Google Cloud Secret Manager

# Install collection
ansible-galaxy collection install google.cloud

# Playbook using GCP Secret Manager
---
- name: Fetch secrets from GCP Secret Manager
  hosts: localhost
  vars:
    gcp_project: my-project-id
    gcp_cred_file: /path/to/service-account.json

  tasks:
    - name: Get secret value
      set_fact:
        db_password: "{{ lookup('google.cloud.gcp_secret_manager',
                        'database-password',
                        project=gcp_project,
                        credentials_file=gcp_cred_file,
                        version='latest') }}"

    - name: Get specific version
      set_fact:
        api_key: "{{ lookup('google.cloud.gcp_secret_manager',
                    'api-key',
                    project=gcp_project,
                    version='3') }}"

    - name: Configure application
      template:
        src: config.j2
        dest: /etc/app/config.yml
      vars:
        database_password: "{{ db_password }}"
        api_key: "{{ api_key }}"
      no_log: true

CI/CD Integration Patterns

GitHub Actions with Ansible Vault

# .github/workflows/deploy.yml
name: Deploy Application

on:
  push:
    branches: [main]
  workflow_dispatch:

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install Ansible
        run: |
          pip install ansible

      - name: Setup Vault Password
        run: |
          echo "${{ secrets.ANSIBLE_VAULT_PASSWORD }}" > .vault_pass
          chmod 600 .vault_pass

      - name: Run Ansible Playbook
        run: |
          ansible-playbook -i inventory/production site.yml \
            --vault-password-file .vault_pass
        env:
          ANSIBLE_HOST_KEY_CHECKING: False

      - name: Cleanup
        if: always()
        run: |
          rm -f .vault_pass

GitLab CI/CD with Multiple Vault IDs

# .gitlab-ci.yml
stages:
  - validate
  - deploy_staging
  - deploy_production

variables:
  ANSIBLE_HOST_KEY_CHECKING: "False"
  ANSIBLE_FORCE_COLOR: "true"

before_script:
  - pip3 install ansible
  - chmod 600 $SSH_PRIVATE_KEY

validate:
  stage: validate
  script:
    - ansible-lint playbooks/
    - ansible-playbook --syntax-check site.yml

deploy_staging:
  stage: deploy_staging
  environment:
    name: staging
  script:
    - echo "$VAULT_PASS_STAGING" > .vault_pass_staging
    - chmod 600 .vault_pass_staging
    - ansible-playbook -i inventory/staging site.yml
        --vault-id staging@.vault_pass_staging
  after_script:
    - rm -f .vault_pass_staging
  only:
    - develop

deploy_production:
  stage: deploy_production
  environment:
    name: production
  script:
    - echo "$VAULT_PASS_PROD" > .vault_pass_prod
    - chmod 600 .vault_pass_prod
    - ansible-playbook -i inventory/production site.yml
        --vault-id prod@.vault_pass_prod
        --check  # Dry run first
    - ansible-playbook -i inventory/production site.yml
        --vault-id prod@.vault_pass_prod
  after_script:
    - rm -f .vault_pass_prod
  only:
    - main
  when: manual

Jenkins Pipeline with Vault

// Jenkinsfile
pipeline {
    agent any

    environment {
        ANSIBLE_HOST_KEY_CHECKING = 'False'
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Validate') {
            steps {
                sh '''
                    ansible-lint playbooks/
                    ansible-playbook --syntax-check site.yml
                '''
            }
        }

        stage('Deploy') {
            steps {
                withCredentials([
                    string(credentialsId: 'ansible-vault-password',
                           variable: 'VAULT_PASS')
                ]) {
                    sh '''
                        echo "$VAULT_PASS" > .vault_pass
                        chmod 600 .vault_pass

                        ansible-playbook -i inventory/production site.yml \
                            --vault-password-file .vault_pass

                        rm -f .vault_pass
                    '''
                }
            }
        }
    }

    post {
        always {
            sh 'rm -f .vault_pass'
        }
    }
}

Azure DevOps Pipeline

# azure-pipelines.yml
trigger:
  - main

pool:
  vmImage: 'ubuntu-latest'

variables:
  - group: ansible-vault-secrets  # Variable group in Azure DevOps

steps:
  - task: UsePythonVersion@0
    inputs:
      versionSpec: '3.11'

  - script: |
      pip install ansible ansible-lint
    displayName: 'Install Ansible'

  - script: |
      ansible-lint playbooks/
      ansible-playbook --syntax-check site.yml
    displayName: 'Validate Playbooks'

  - script: |
      echo "$(vaultPassword)" > .vault_pass
      chmod 600 .vault_pass
    displayName: 'Setup Vault Password'

  - script: |
      ansible-playbook -i inventory/production site.yml \
        --vault-password-file .vault_pass
    displayName: 'Deploy Application'
    env:
      ANSIBLE_HOST_KEY_CHECKING: False

  - script: |
      rm -f .vault_pass
    displayName: 'Cleanup'
    condition: always()

Team Collaboration Strategies

Multi-Team Vault Strategy

# Directory structure for multiple teams
ansible-project/
├── group_vars/
│   ├── all/
│   │   ├── vars.yml
│   │   └── vault.yml          # Common secrets (vault-id: common)
│   ├── production/
│   │   ├── vars.yml
│   │   └── vault.yml          # Prod secrets (vault-id: prod)
│   └── development/
│       ├── vars.yml
│       └── vault.yml          # Dev secrets (vault-id: dev)
├── team_vars/
│   ├── devops/
│   │   └── vault.yml          # DevOps team secrets (vault-id: devops)
│   ├── security/
│   │   └── vault.yml          # Security team secrets (vault-id: security)
│   └── database/
│       └── vault.yml          # DBA team secrets (vault-id: database)
└── .vault_pass.sh             # Script to manage multiple passwords

# Encrypt with specific team vault
ansible-vault create team_vars/devops/vault.yml --vault-id devops@prompt

# Run with multiple vault IDs
ansible-playbook site.yml \
  --vault-id common@.vault_pass_common \
  --vault-id prod@.vault_pass_prod \
  --vault-id devops@.vault_pass_devops

Vault Password Management Script

#!/usr/bin/env bash
# .vault_pass.sh - Centralized vault password management

set -euo pipefail

VAULT_ID="${1:-default}"
VAULT_PASS_DIR="${HOME}/.ansible/vault_passwords"

# Ensure directory exists with secure permissions
mkdir -p "${VAULT_PASS_DIR}"
chmod 700 "${VAULT_PASS_DIR}"

# Map vault IDs to password sources
case "${VAULT_ID}" in
    "common")
        # From environment variable
        echo "${ANSIBLE_VAULT_PASS_COMMON:-}"
        ;;
    "prod")
        # From 1Password CLI
        op read "op://Production/Ansible-Vault/password"
        ;;
    "dev")
        # From file
        cat "${VAULT_PASS_DIR}/dev.key"
        ;;
    "devops")
        # From AWS Secrets Manager
        aws secretsmanager get-secret-value \
            --secret-id ansible/vault/devops \
            --query SecretString \
            --output text
        ;;
    *)
        echo "Unknown vault ID: ${VAULT_ID}" >&2
        exit 1
        ;;
esac

# Usage in ansible.cfg:
# [defaults]
# vault_identity_list = common@./.vault_pass.sh, prod@./.vault_pass.sh

Role-Based Access with Vault IDs

# ansible.cfg
[defaults]
vault_identity_list = common@prompt, prod@~/.ansible/vault_prod, dev@~/.ansible/vault_dev

# Team access matrix:
#
# Vault ID    | Developers | DevOps | Security | DBAs
# ------------|-----------|--------|----------|------
# common      | ✓         | ✓      | ✓        | ✓
# dev         | ✓         | ✓      | ✓        | ✓
# staging     | ✗         | ✓      | ✓        | ✓
# prod        | ✗         | ✓      | ✓        | ✓
# database    | ✗         | ✗      | ✓        | ✓
# security    | ✗         | ✗      | ✓        | ✗

# Playbook with conditional vault loading
---
- name: Deploy application
  hosts: all
  vars_files:
    - group_vars/all/vault.yml  # Common secrets (all teams)
    - "{{ 'group_vars/' + env + '/vault.yml' }}"  # Environment secrets
    - "{{ 'team_vars/' + team + '/vault.yml' if team_vault_required else [] }}"

  tasks:
    - name: Verify access to required secrets
      assert:
        that:
          - common_secret is defined
          - env_secret is defined
        msg: "Missing required secrets - check vault access"

Automated Vault Management

Bulk Secret Encryption

#!/usr/bin/env python3
# encrypt_secrets.py - Bulk encrypt secrets from JSON

import json
import subprocess
import sys

def encrypt_string(value, vault_id, vault_pass_file):
    """Encrypt a string using ansible-vault"""
    cmd = [
        'ansible-vault', 'encrypt_string',
        '--vault-id', f'{vault_id}@{vault_pass_file}',
        '--stdin-name', 'encrypted'
    ]

    result = subprocess.run(
        cmd,
        input=value.encode(),
        capture_output=True,
        text=False
    )

    if result.returncode != 0:
        raise Exception(f"Encryption failed: {result.stderr}")

    return result.stdout.decode()

def main():
    # Load secrets from JSON (e.g., from secret manager)
    with open('secrets.json', 'r') as f:
        secrets = json.load(f)

    output = "---\n"

    for key, value in secrets.items():
        print(f"Encrypting {key}...", file=sys.stderr)
        encrypted = encrypt_string(
            str(value),
            vault_id='prod',
            vault_pass_file='.vault_pass'
        )

        # Extract just the vault blob
        vault_lines = [l for l in encrypted.split('\n')
                      if l and not l.startswith('encrypted:')]

        output += f"{key}: !vault |\n"
        output += '\n'.join(f"  {line}" for line in vault_lines)
        output += "\n"

    # Write to vault file
    with open('group_vars/production/vault.yml', 'w') as f:
        f.write(output)

    print("✓ All secrets encrypted successfully", file=sys.stderr)

if __name__ == '__main__':
    main()

# Usage:
# echo '{"db_password": "secret123", "api_key": "key456"}' > secrets.json
# python3 encrypt_secrets.py

Secret Rotation Automation

#!/usr/bin/env bash
# rotate_secrets.sh - Automated secret rotation

set -euo pipefail

OLD_VAULT_PASS=".vault_pass.old"
NEW_VAULT_PASS=".vault_pass.new"
VAULT_FILES=(
    "group_vars/production/vault.yml"
    "group_vars/staging/vault.yml"
    "host_vars/db01/vault.yml"
)

# Generate new vault password
generate_password() {
    openssl rand -base64 32
}

# Backup current vault files
backup_vaults() {
    local timestamp=$(date +%Y%m%d_%H%M%S)
    local backup_dir="vault_backup_${timestamp}"
    mkdir -p "${backup_dir}"

    for vault_file in "${VAULT_FILES[@]}"; do
        if [[ -f "${vault_file}" ]]; then
            cp "${vault_file}" "${backup_dir}/"
        fi
    done

    echo "✓ Backup created: ${backup_dir}"
}

# Rekey all vault files
rekey_vaults() {
    for vault_file in "${VAULT_FILES[@]}"; do
        if [[ -f "${vault_file}" ]]; then
            echo "Rekeying ${vault_file}..."
            ansible-vault rekey \
                --vault-password-file="${OLD_VAULT_PASS}" \
                --new-vault-password-file="${NEW_VAULT_PASS}" \
                "${vault_file}"
        fi
    done
}

# Main rotation process
main() {
    echo "=== Ansible Vault Password Rotation ==="

    # Backup current password
    if [[ ! -f ".vault_pass" ]]; then
        echo "Error: .vault_pass not found"
        exit 1
    fi

    cp .vault_pass "${OLD_VAULT_PASS}"

    # Generate and save new password
    generate_password > "${NEW_VAULT_PASS}"
    chmod 600 "${NEW_VAULT_PASS}"

    # Backup vault files
    backup_vaults

    # Perform rotation
    rekey_vaults

    # Replace old password with new
    mv "${NEW_VAULT_PASS}" .vault_pass

    # Update in external secret manager (example: AWS Secrets Manager)
    aws secretsmanager update-secret \
        --secret-id ansible/vault/production \
        --secret-string file://.vault_pass

    echo "✓ Vault password rotation complete"
    echo "⚠ Don't forget to update the password in CI/CD systems!"

    # Cleanup
    rm -f "${OLD_VAULT_PASS}"
}

main "$@"

Migration Strategies

Migrating from Plain Text to Vault

#!/usr/bin/env python3
# migrate_to_vault.py - Convert plain text vars to vault

import yaml
import subprocess
import os
import re

SENSITIVE_PATTERNS = [
    r'password', r'passwd', r'pwd',
    r'secret', r'api[_-]?key', r'token',
    r'private[_-]?key', r'cert',
]

def is_sensitive(key):
    """Check if variable name indicates sensitive data"""
    key_lower = key.lower()
    return any(re.search(pattern, key_lower) for pattern in SENSITIVE_PATTERNS)

def encrypt_value(value, name, vault_pass_file):
    """Encrypt a single value"""
    cmd = [
        'ansible-vault', 'encrypt_string',
        '--vault-password-file', vault_pass_file,
        '--stdin-name', name,
        str(value)
    ]

    result = subprocess.run(cmd, capture_output=True, text=True)
    return result.stdout

def migrate_file(input_file, output_vault_file, output_vars_file, vault_pass_file):
    """Migrate sensitive vars to encrypted vault file"""

    with open(input_file, 'r') as f:
        data = yaml.safe_load(f) or {}

    vault_vars = {}
    regular_vars = {}

    for key, value in data.items():
        if is_sensitive(key):
            # Move to vault with vault_ prefix
            vault_key = f"vault_{key}" if not key.startswith('vault_') else key
            vault_vars[vault_key] = value
            # Reference in regular vars
            regular_vars[key] = f"{{{{ {vault_key} }}}}"
        else:
            regular_vars[key] = value

    # Write vault file (will be encrypted)
    with open(output_vault_file, 'w') as f:
        yaml.dump(vault_vars, f, default_flow_style=False)

    # Encrypt vault file
    subprocess.run([
        'ansible-vault', 'encrypt',
        '--vault-password-file', vault_pass_file,
        output_vault_file
    ])

    # Write regular vars file
    with open(output_vars_file, 'w') as f:
        f.write('---\n')
        for key, value in regular_vars.items():
            if isinstance(value, str) and value.startswith('{{'):
                f.write(f'{key}: "{value}"\n')
            else:
                yaml.dump({key: value}, f, default_flow_style=False)

    print(f"✓ Migrated {len(vault_vars)} sensitive vars to {output_vault_file}")
    print(f"✓ Wrote {len(regular_vars)} vars to {output_vars_file}")

# Usage:
# python3 migrate_to_vault.py \
#     group_vars/production/all.yml \
#     group_vars/production/vault.yml \
#     group_vars/production/vars.yml \
#     .vault_pass

Migrating Between Vault Providers

---
# migrate_vault_provider.yml
# Migrate from Ansible Vault to HashiCorp Vault

- name: Migrate secrets to HashiCorp Vault
  hosts: localhost
  vars:
    ansible_vault_file: group_vars/production/vault.yml
    hashi_vault_url: https://vault.example.com:8200
    hashi_vault_token: "{{ lookup('env', 'VAULT_TOKEN') }}"
    hashi_vault_path: secret/data/ansible/production

  tasks:
    - name: Load encrypted Ansible Vault file
      include_vars:
        file: "{{ ansible_vault_file }}"
      no_log: true

    - name: Get all vault variables
      set_fact:
        vault_secrets: "{{ vars | dict2items | selectattr('key', 'match', '^vault_.*') }}"

    - name: Write secrets to HashiCorp Vault
      uri:
        url: "{{ hashi_vault_url }}/v1/{{ hashi_vault_path }}"
        method: POST
        headers:
          X-Vault-Token: "{{ hashi_vault_token }}"
        body_format: json
        body:
          data: "{{ dict(vault_secrets | map(attribute='key') | zip(vault_secrets | map(attribute='value'))) }}"
        status_code: [200, 204]
      no_log: true

    - name: Verify migration
      uri:
        url: "{{ hashi_vault_url }}/v1/{{ hashi_vault_path }}"
        headers:
          X-Vault-Token: "{{ hashi_vault_token }}"
      register: verify_result

    - name: Report migration status
      debug:
        msg: "✓ Migrated {{ vault_secrets | length }} secrets to HashiCorp Vault"

Performance Optimization

Caching Vault Passwords

# ansible.cfg
[defaults]
# Cache vault password for duration of playbook execution
vault_password_file = ~/.ansible/vault_pass.sh

# vault_pass.sh - with caching
#!/usr/bin/env bash

CACHE_FILE="/tmp/.ansible_vault_cache_$$"
CACHE_TTL=3600  # 1 hour

if [[ -f "${CACHE_FILE}" ]]; then
    CACHE_AGE=$(($(date +%s) - $(stat -f %m "${CACHE_FILE}" 2>/dev/null || stat -c %Y "${CACHE_FILE}")))
    if [[ ${CACHE_AGE} -lt ${CACHE_TTL} ]]; then
        cat "${CACHE_FILE}"
        exit 0
    fi
fi

# Fetch password (expensive operation)
PASSWORD=$(fetch_password_from_1password)

# Cache it
echo "${PASSWORD}" > "${CACHE_FILE}"
chmod 600 "${CACHE_FILE}"

echo "${PASSWORD}"

Parallel Vault Operations

---
# Optimize vault operations with parallel execution
- name: Deploy with parallel secret fetching
  hosts: app_servers
  gather_facts: no

  tasks:
    # Fetch all secrets in parallel
    - name: Fetch secrets concurrently
      async: 30
      poll: 0
      set_fact:
        "{{ item.name }}": "{{ lookup('community.hashi_vault.hashi_vault', item.path) }}"
      loop:
        - { name: 'db_password', path: 'secret/data/db:password' }
        - { name: 'api_key', path: 'secret/data/api:key' }
        - { name: 'ssl_cert', path: 'secret/data/ssl:certificate' }
        - { name: 'ssl_key', path: 'secret/data/ssl:private_key' }
      register: secret_jobs

    # Wait for all secrets to be fetched
    - name: Wait for secret fetching
      async_status:
        jid: "{{ item.ansible_job_id }}"
      register: secret_results
      until: secret_results.finished
      retries: 10
      loop: "{{ secret_jobs.results }}"

    # Continue with deployment
    - name: Deploy application
      template:
        src: app_config.j2
        dest: /etc/app/config.yml
      vars:
        database_password: "{{ db_password }}"
        api_key: "{{ api_key }}"

Real-World Use Cases

Multi-Environment Deployment

# Directory structure
ansible-project/
├── ansible.cfg
├── inventory/
│   ├── dev.yml
│   ├── staging.yml
│   └── production.yml
├── group_vars/
│   ├── all/
│   │   ├── vars.yml
│   │   └── vault.yml              # Common secrets (vault-id: common)
│   ├── dev/
│   │   ├── vars.yml
│   │   └── vault.yml              # Dev secrets (vault-id: dev)
│   ├── staging/
│   │   ├── vars.yml
│   │   └── vault.yml              # Staging secrets (vault-id: staging)
│   └── production/
│       ├── vars.yml
│       └── vault.yml              # Production secrets (vault-id: prod)
└── deploy.sh

# deploy.sh - Smart deployment script
#!/usr/bin/env bash

ENVIRONMENT="${1:-dev}"

case "${ENVIRONMENT}" in
    "dev")
        VAULT_IDS="common@.vault_pass_common dev@.vault_pass_dev"
        INVENTORY="inventory/dev.yml"
        ;;
    "staging")
        VAULT_IDS="common@.vault_pass_common staging@.vault_pass_staging"
        INVENTORY="inventory/staging.yml"
        ;;
    "production")
        VAULT_IDS="common@.vault_pass_common prod@prompt"
        INVENTORY="inventory/production.yml"
        read -p "⚠ Deploying to PRODUCTION. Continue? (yes/no) " confirm
        [[ "${confirm}" != "yes" ]] && exit 1
        ;;
    *)
        echo "Unknown environment: ${ENVIRONMENT}"
        exit 1
        ;;
esac

ansible-playbook -i "${INVENTORY}" site.yml \
    $(echo "${VAULT_IDS}" | xargs -n1 echo --vault-id)

Database Credential Rotation

---
# rotate_db_credentials.yml
- name: Rotate database credentials
  hosts: localhost
  vars:
    vault_file: group_vars/production/vault.yml
    db_hosts:
      - prod-db-01.example.com
      - prod-db-02.example.com

  tasks:
    - name: Generate new database password
      set_fact:
        new_db_password: "{{ lookup('password', '/dev/null length=32 chars=ascii_letters,digits,punctuation') }}"
      no_log: true

    - name: Update password in databases
      mysql_user:
        login_host: "{{ item }}"
        login_user: root
        login_password: "{{ vault_mysql_root_password }}"
        name: app_user
        password: "{{ new_db_password }}"
        state: present
      loop: "{{ db_hosts }}"
      no_log: true

    - name: Update vault file with new password
      lineinfile:
        path: "{{ vault_file }}"
        regexp: '^vault_db_password:'
        line: "vault_db_password: {{ new_db_password }}"
      delegate_to: localhost
      no_log: true

    - name: Re-encrypt vault file
      command: >
        ansible-vault encrypt {{ vault_file }}
        --vault-password-file .vault_pass
      delegate_to: localhost

    - name: Restart application servers
      service:
        name: myapp
        state: restarted
      delegate_to: "{{ item }}"
      loop: "{{ groups['app_servers'] }}"

    - name: Verify database connectivity
      shell: >
        mysql -h {{ item }} -u app_user -p{{ new_db_password }}
        -e "SELECT 1"
      loop: "{{ db_hosts }}"
      no_log: true
      changed_when: false

Secrets Audit Trail

---
# audit_vault_access.yml
- name: Audit vault access and usage
  hosts: localhost
  tasks:
    - name: List all encrypted files
      find:
        paths: "{{ playbook_dir }}"
        patterns: "*.yml"
        contains: "\\$ANSIBLE_VAULT"
        recurse: yes
      register: vault_files

    - name: Get file metadata
      stat:
        path: "{{ item.path }}"
      loop: "{{ vault_files.files }}"
      register: file_stats

    - name: Generate audit report
      template:
        src: vault_audit_report.j2
        dest: "/tmp/vault_audit_{{ ansible_date_time.date }}.html"
      vars:
        total_vault_files: "{{ vault_files.matched }}"
        files_details: "{{ file_stats.results }}"

    - name: Check for secrets in git history
      shell: |
        git grep -i "password\|api.key\|secret" $(git rev-list --all) -- "*.yml" || true
      register: git_secrets_check
      changed_when: false

    - name: Report potential leaked secrets
      debug:
        msg: "⚠ Found {{ git_secrets_check.stdout_lines | length }} potential secrets in git history"
      when: git_secrets_check.stdout_lines | length > 0

Advanced Troubleshooting

Vault Debugging Techniques

# Enable verbose vault debugging
export ANSIBLE_DEBUG=1
ansible-playbook site.yml --ask-vault-pass -vvvv 2>&1 | grep -i vault

# Check vault file integrity
ansible-vault view group_vars/production/vault.yml --vault-password-file .vault_pass

# Test vault password without running playbook
echo "Test" | ansible-vault encrypt_string --vault-password-file .vault_pass --stdin-name test

# Verify vault can be decrypted
ansible-vault decrypt group_vars/production/vault.yml --output=- | head -5

# Check vault ID in encrypted file
head -1 group_vars/production/vault.yml
# Should show: $ANSIBLE_VAULT;1.1;AES256 (no vault ID) or
#              $ANSIBLE_VAULT;1.2;AES256;vault_id_name

# List all variables (including vault vars) for a host
ansible -i inventory/production -m debug -a "var=hostvars[inventory_hostname]" web01

Common Vault Errors and Solutions

# Error: Decryption failed (no vault secrets were found)
# Solution 1: Wrong password
ansible-playbook site.yml --ask-vault-pass  # Re-enter password

# Solution 2: Wrong vault ID
ansible-vault view --vault-id prod@prompt group_vars/production/vault.yml

# Solution 3: File not actually encrypted
head -1 suspicious_file.yml  # Should start with $ANSIBLE_VAULT

# Error: Vault format unhexlify error
# Cause: File corrupted or partially encrypted
# Solution: Restore from backup
git checkout group_vars/production/vault.yml

# Error: Conflicting action statements: decrypt, encrypt
# Cause: Trying to encrypt already encrypted file
# Solution: Check if already encrypted
head -1 file.yml

# Error: ERROR! vars file group_vars/prod/vault.yml has vault value
# Cause: Trying to include vault file as regular vars
# Solution: Ensure vault password is provided
ansible-playbook site.yml --vault-password-file .vault_pass

# Error: Found variable using reserved name: vault_*
# Not actually an error - vault_ prefix is convention, not reserved
# Can safely ignore or rename if desired

Vault Recovery Procedures

#!/usr/bin/env bash
# vault_recovery.sh - Emergency vault recovery

# Scenario 1: Lost vault password but have plaintext backup
recover_from_plaintext() {
    local plaintext_backup="$1"
    local vault_file="$2"
    local new_password="$3"

    # Re-encrypt with new password
    echo "${new_password}" | ansible-vault encrypt \
        --vault-password-file=/dev/stdin \
        "${plaintext_backup}" \
        --output="${vault_file}"
}

# Scenario 2: Corrupted vault file but have git history
recover_from_git() {
    local vault_file="$1"
    local backup_file="${vault_file}.backup"

    # Find last known good commit
    git log --all --full-history -- "${vault_file}"

    # Checkout specific version
    read -p "Enter commit hash: " commit_hash
    git show "${commit_hash}:${vault_file}" > "${backup_file}"

    # Verify it's valid
    ansible-vault view "${backup_file}"
}

# Scenario 3: Need to extract secrets without password
# (requires access to Ansible vault key derivation)
brute_force_warning() {
    cat <

# Create and encrypt
ansible-vault create secrets.yml
ansible-vault encrypt existing.yml
ansible-vault encrypt_string 'secret' --name 'var_name'

# View and edit
ansible-vault view secrets.yml
ansible-vault edit secrets.yml

# Decrypt and rekey
ansible-vault decrypt secrets.yml
ansible-vault rekey secrets.yml

# Run playbooks
ansible-playbook site.yml --ask-vault-pass
ansible-playbook site.yml --vault-password-file .vault_pass
ansible-playbook site.yml --vault-id dev@prompt --vault-id prod@.vault_pass

# Multiple vault IDs
ansible-vault create --vault-id prod@prompt secrets_prod.yml
ansible-vault edit --vault-id dev@.vault_pass_dev secrets_dev.yml
ansible-playbook site.yml --vault-id dev@prompt --vault-id prod@prompt

# External secret managers
ansible-playbook site.yml -e @<(./fetch_secrets_from_vault.sh)

# Debug
head -1 file.yml  # Check if encrypted
ansible-vault view file.yml --vault-password-file .vault_pass
ansible-playbook site.yml --ask-vault-pass -vvvv