AWS Config Rules & Compliance Automation
Implement automated compliance monitoring with AWS Config Rules, custom evaluations, and remediation actions for continuous governance.
📋 Prerequisites
- In-depth knowledge of AWS Config, IAM roles, and S3.
- Proficiency in a programming language for Lambda (Python used in examples).
- Experience with Infrastructure as Code (Terraform/CloudFormation) for deployment.
- Understanding of AWS Systems Manager (SSM) Automation documents.
- Access to an AWS Organization for multi-account patterns.
💡 From Periodic Audits to Continuous Compliance
Manual compliance checks are slow, error-prone, and only provide a point-in-time snapshot. AWS Config transforms compliance into a continuous, automated process. It allows you to codify your compliance requirements, continuously monitor for deviations, and even trigger automated remediations to self-heal your environment in real-time.
What You'll Learn
🏷️ Topics Covered
Advanced Multi-Account Config Architecture
In an enterprise environment, you don't manage compliance for a single account. The goal is to gain a unified view across your entire AWS Organization. This is achieved using a central aggregator model, where a designated "Audit" or "Security" account collects compliance data from all member accounts.
🎯 The Aggregator Pattern
This pattern involves enabling the AWS Config recorder in all member accounts and regions, and then creating a Configuration Aggregator in your central audit account. The aggregator pulls in resource configurations and compliance statuses, providing a single dashboard for organization-wide visibility.
Core Aggregator Components
📝 Member Accounts
Each account in the organization runs a Configuration Recorder to track all resource changes.
📦 Delivery Channel
A centralized S3 bucket in the Audit account where all configuration snapshots and history files are delivered.
🔄 Configuration Aggregator
A resource in the Audit account that collects Config data from specified source accounts and regions.
📊 Centralized Dashboard
A single pane of glass in the Audit account to view compliance status, resource inventory, and configuration history for the entire organization.
Managed vs. Custom Config Rules
AWS provides a library of managed rules for common compliance checks. While powerful, they can't cover every specific organizational requirement. For that, you need custom rules.
Managed Rules
- Maintained by AWS.
- Easy to enable and configure.
- Covers common best practices (e.g., EBS encryption, public S3 buckets).
- Limited customization options.
- Best for standard security and operational checks.
Custom Rules
- Developed and maintained by you.
- Infinitely flexible; can check any resource property or configuration.
- Implemented with AWS Lambda functions.
- Requires coding and testing effort.
- Essential for organization-specific policies (e.g., tagging, naming conventions).
Developing Custom Rules with AWS Lambda
Custom rules give you the power to define any compliance check imaginable. The rule is powered by a Lambda function that contains the evaluation logic. AWS Config invokes this function whenever a relevant resource is created or changed.
Example: Enforcing a Mandatory "CostCenter" Tag
This advanced rule checks that all EC2 instances and EBS volumes have a `CostCenter` tag. If the tag is missing or its value is empty, the resource is marked as non-compliant.
🐍 Python: Custom Tag Compliance Lambda Rule
import json
import boto3
config_client = boto3.client('config')
def evaluate_compliance(configuration_item):
"""Evaluates if a resource has a non-empty CostCenter tag."""
if 'tags' not in configuration_item or not configuration_item['tags']:
return 'NON_COMPLIANT'
cost_center_tag = next((tag for tag in configuration_item['tags'] if tag['key'] == 'CostCenter'), None)
if not cost_center_tag or not cost_center_tag['value']:
return 'NON_COMPLIANT'
return 'COMPLIANT'
def lambda_handler(event, context):
invoking_event = json.loads(event['invokingEvent'])
configuration_item = invoking_event['configurationItem']
# Only evaluate EC2 Instances and EBS Volumes
supported_resource_types = ['AWS::EC2::Instance', 'AWS::EC2::Volume']
if configuration_item['resourceType'] not in supported_resource_types:
return
compliance_status = evaluate_compliance(configuration_item)
evaluation = {
'ComplianceResourceType': configuration_item['resourceType'],
'ComplianceResourceId': configuration_item['resourceId'],
'ComplianceType': compliance_status,
'Annotation': f"Resource must have a non-empty 'CostCenter' tag.",
'OrderingTimestamp': configuration_item['configurationItemCaptureTime']
}
try:
response = config_client.put_evaluations(
Evaluations=[evaluation],
ResultToken=event['resultToken']
)
print("Successfully put evaluation results.")
except Exception as e:
print(f"Error putting evaluations: {e}")
raise eAutomated Remediation with SSM Automation
Detecting non-compliance is only half the battle. True continuous governance involves automatically fixing the issue. You can configure AWS Config to trigger an SSM Automation document as a remediation action for a non-compliant rule.
Example: Automatically Stopping Non-Compliant EC2 Instances
This SSM Automation document will be triggered when the custom rule above finds an EC2 instance to be non-compliant. Its action is to stop the instance to prevent it from incurring costs without proper allocation.
⚙️ YAML: SSM Automation Document for Remediation
description: >-
Stops an EC2 instance that is found to be non-compliant by an AWS Config rule.
schemaVersion: '0.3'
assumeRole: '{{ AutomationAssumeRole }}'
parameters:
AutomationAssumeRole:
type: String
description: (Required) The ARN of the role that allows Automation to perform the actions on your behalf.
InstanceId:
type: String
description: (Required) The ID of the EC2 instance to stop.
mainSteps:
- name: CheckInstanceState
action: 'aws:assertAwsResourceProperty'
inputs:
Service: ec2
Api: DescribeInstances
InstanceIds:
- '{{ InstanceId }}'
PropertySelector: '$.Reservations[0].Instances[0].State.Name'
DesiredValues:
- running
onFailure: Abort
maxAttempts: 2
timeoutSeconds: 60
- name: StopInstance
action: 'aws:executeAwsApi'
inputs:
Service: ec2
Api: StopInstances
InstanceIds:
- '{{ InstanceId }}'
description: Stops the non-compliant EC2 instance.
- name: VerifyInstanceStopped
action: 'aws:waitForAwsResourceProperty'
timeoutSeconds: 300
inputs:
Service: ec2
Api: DescribeInstances
InstanceIds:
- '{{ InstanceId }}'
PropertySelector: '$.Reservations[0].Instances[0].State.Name'
DesiredValues:
- stopped
onFailure: AbortDeploying Compliance-as-Code with Terraform
To manage compliance at scale, you must define your rules, Lambda functions, and remediation actions as code. This ensures they are versioned, auditable, and consistently deployed across your organization.
🏗️ HCL: Deploying a Custom Rule and Remediation with Terraform
# 1. IAM Role for the Lambda function
resource "aws_iam_role" "config_lambda_role" {
name = "config-custom-tag-rule-role"
assume_role_policy = jsonencode({
Version = "2012-10-17",
Statement = [{
Action = "sts:AssumeRole",
Effect = "Allow",
Principal = { Service = "lambda.amazonaws.com" }
}]
})
}
resource "aws_iam_role_policy_attachment" "config_lambda_policy" {
role = aws_iam_role.config_lambda_role.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSConfigRulesExecutionRole"
}
# 2. Lambda function for the custom rule
data "archive_file" "lambda_zip" {
type = "zip"
source_file = "lambda_function.py"
output_path = "lambda_function.zip"
}
resource "aws_lambda_function" "tag_compliance_rule" {
function_name = "CustomTagComplianceRule"
handler = "lambda_function.lambda_handler"
role = aws_iam_role.config_lambda_role.arn
runtime = "python3.9"
filename = data.archive_file.lambda_zip.output_path
source_code_hash = data.archive_file.lambda_zip.output_base64sha256
}
# 3. The Custom AWS Config Rule
resource "aws_config_custom_rule" "enforce_costcenter_tag" {
name = "enforce-costcenter-tag"
lambda_function_arn = aws_lambda_function.tag_compliance_rule.arn
trigger_types = ["ConfigurationItemChangeNotification"]
source {
owner = "CUSTOM_LAMBDA"
source_identifier = aws_lambda_function.tag_compliance_rule.arn
}
scope {
compliance_resource_types = ["AWS::EC2::Instance", "AWS::EC2::Volume"]
}
}
# 4. The Remediation Configuration linking the rule to the SSM document
resource "aws_config_remediation_configuration" "stop_noncompliant_ec2" {
config_rule_name = aws_config_custom_rule.enforce_costcenter_tag.name
target_id = "AWS-StopEC2Instance" # Pre-existing SSM Document
target_type = "SSM_DOCUMENT"
automatic = true
maximum_automatic_attempts = 5
retry_attempt_seconds = 60
parameter {
name = "InstanceId"
resource_value {
value = "RESOURCE_ID"
}
}
}Enterprise Best Practices and Strategies
🔑 AWS Config Advanced Best Practices
- Centralize Everything: Use Configuration Aggregators in a dedicated audit account for organization-wide visibility.
- Embrace Compliance-as-Code: Define all rules and remediations in IaC (Terraform/CloudFormation) and manage them via Git and CI/CD.
- Test Remediations Carefully: Before enabling automatic remediation, test SSM documents thoroughly. Start with manual approvals or run in a non-production OU.
- Develop a Phased Rollout: When introducing a new, strict rule, deploy it in "detection-only" mode first to assess its impact before enabling remediation.
- Use Conformance Packs: For standard frameworks (like PCI-DSS, HIPAA), use AWS Config Conformance Packs to deploy a collection of managed rules and remediations with a single template.
- Integrate with Security Hub: Forward all AWS Config findings to AWS Security Hub to correlate them with other security data from services like GuardDuty and Inspector.
- Optimize for Cost: Be mindful that each configuration change tracked incurs a small cost. For highly dynamic environments, consider which resources are essential to track.