Backup Best Practices

Consistency: Take all three backups in quick succession to ensure temporal consistency. Ideally, perform backups during a maintenance window with minimal activity.

Verification: After creating backups, verify file sizes are reasonable and files are not corrupted:

# Verify PostgreSQL backup
gunzip -t postgres-backup-*.sql.gz

# Verify Elasticsearch backup
tar -tzf elasticsearch-backup-*.tar.gz | head

# Verify Consul backup (should be a valid binary file)
file consul-backup-*.snap

Storage: Store backup files in a secure location separate from the source cluster. Consider:

  • Copying to a backup server or NAS

  • Uploading to cloud storage (S3, Azure Blob, etc.)

  • Maintaining multiple backup generations

  • Encrypting backups if they contain sensitive data

Documentation: Record the backup time stamp, source cluster version, and any special circumstances:

# Create a backup manifest
cat > backup-manifest.txt << EOF
Backup Date: $(date)
K3s Version: $(kubectl version --short)
Product Version: $(kubectl get apps -o wide)
PostgreSQL Backup: postgres-backup-*.sql.gz
Elasticsearch Backup: elasticsearch-backup-*.tar.gz
Consul Backup: consul-backup-*.snap
EOF