Threat Detection & Response
Implement comprehensive AWS security monitoring with Security Hub, GuardDuty, and automated incident response workflows for threat detection and compliance.
📋 Prerequisites
- Expert knowledge of GuardDuty, Security Hub, IAM, and EventBridge.
- Advanced proficiency with AWS Step Functions and Lambda (Python).
- Strong experience with Infrastructure as Code (Terraform).
- An AWS Organization with a delegated security administrator account.
💡 From Alerting to Orchestration: The Modern SecOps Workflow
Cloud threat detection is no longer about just receiving alerts. A modern Security Operations (SecOps) practice in AWS revolves around automated orchestration. **GuardDuty** provides the intelligent threat detection, **Security Hub** acts as the central nervous system to aggregate and prioritize findings, and services like **Step Functions** and **Lambda** execute robust, repeatable incident response playbooks, minimizing Mean Time to Response (MTTR) and human error.
What You'll Learn
🏷️ Topics Covered
Advanced GuardDuty Configuration
Beyond the default setup, you can significantly enhance GuardDuty's effectiveness by integrating your own intelligence and managing findings programmatically.
🗂️ HCL: Managing GuardDuty with Terraform
This Terraform code enables GuardDuty for an entire organization via a delegated administrator account and configures S3 protection. It also shows how to upload a custom threat intelligence set of malicious IPs.
# Run in the Management Account to set the delegated admin
resource "aws_guardduty_organization_admin_account" "security_admin" {
admin_account_id = "111122223333" # Your Security Account ID
}
# Run in the Delegated Security Account to manage GuardDuty
resource "aws_guardduty_detector" "primary" {
enable = true
finding_publishing_frequency = "FIFTEEN_MINUTES"
datasources {
s3_logs {
enable = true
}
kubernetes {
audit_logs {
enable = true
}
}
}
}
resource "aws_guardduty_organization_configuration" "org_config" {
detector_id = aws_guardduty_detector.primary.id
auto_enable_organization_members = "ALL"
datasources {
s3_logs {
auto_enable = true
}
}
}
# Upload a custom threat intel set of known bad IPs
resource "aws_guardduty_threatintelset" "bad_ips" {
activate = true
detector_id = aws_guardduty_detector.primary.id
format = "TXT"
location = "https://my-threat-intel.s3.us-east-1.amazonaws.com/bad_ips.txt"
name = "MyKnownBadIPs"
}Incident Response Orchestration with Step Functions
While a single Lambda function can work for simple responses, complex playbooks benefit from the resilience and visibility of **AWS Step Functions**. A state machine can coordinate multiple Lambda functions, manage retries, and provide a clear audit trail of the entire response workflow.
Pattern: Isolate, Snapshot, and Analyze
This state machine orchestrates the response to a compromised EC2 instance finding from GuardDuty.
⚙️ JSON: Incident Response Step Functions State Machine
{
"Comment": "Incident Response State Machine for EC2 Findings",
"StartAt": "Isolate_Instance",
"States": {
"Isolate_Instance": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-east-1:111122223333:function:isolate-instance:$LATEST",
"Payload.$": "$"
},
"Retry": [
{
"ErrorEquals": [ "Lambda.ServiceException" ],
"IntervalSeconds": 2,
"MaxAttempts": 3,
"BackoffRate": 2
}
],
"Next": "Create_Forensic_Snapshot",
"Catch": [
{
"ErrorEquals": [ "States.ALL" ],
"Next": "Notify_Failure"
}
]
},
"Create_Forensic_Snapshot": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-east-1:111122223333:function:snapshot-instance:$LATEST",
"Payload.$": "$"
},
"Next": "Enrich_Finding_Data"
},
"Enrich_Finding_Data": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-east-1:111122223333:function:enrich-finding:$LATEST",
"Payload.$": "$"
},
"Next": "Notify_Success"
},
"Notify_Success": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Parameters": {
"TopicArn": "arn:aws:sns:us-east-1:111122223333:SecOps-Alerts",
"Message": {
"Input.$": "$"
}
},
"End": true
},
"Notify_Failure": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Parameters": {
"TopicArn": "arn:aws:sns:us-east-1:111122223333:SecOps-Alerts-Critical",
"Message": {
"Input.$": "$"
}
},
"End": true
}
}
}Proactive Threat Hunting with Amazon Detective
Alerts tell you *what* happened, but not always *how* or *why*. **Amazon Detective** automatically processes logs from CloudTrail, VPC Flow Logs, and GuardDuty to build a linked graph model of your resources. It's an essential tool for root cause analysis and threat hunting.
🕵️ From Finding to Investigation
Within the GuardDuty or Security Hub console, you can pivot directly from a finding to the Amazon Detective console by choosing "Investigate". Detective provides visualizations of the resource's behavior before, during, and after the finding, helping you determine the blast radius and identify the initial point of compromise.
Deploying the Full SecOps Stack as Code
Your entire threat detection and response framework should be codified to ensure it's versioned, auditable, and consistently deployed across your organization.
🏗️ HCL: Deploying the Orchestration Workflow with Terraform
This Terraform configuration deploys the EventBridge rule that triggers the Step Functions state machine in response to high-severity GuardDuty EC2 findings.
# 1. The Step Functions State Machine IAM Role
resource "aws_iam_role" "step_functions_role" {
name = "IncidentResponseStateMachineRole"
assume_role_policy = jsonencode({
Version = "2012-10-17",
Statement = [{
Action = "sts:AssumeRole",
Effect = "Allow",
Principal = { Service = "states.amazonaws.com" }
}]
})
}
# (Policy attachment for this role needs lambda:Invoke and sns:Publish)
# 2. The Step Functions State Machine
resource "aws_sfn_state_machine" "incident_response" {
name = "EC2IncidentResponse"
role_arn = aws_iam_role.step_functions_role.arn
definition = file("state_machines/incident_response.json")
}
# 3. The EventBridge Rule to trigger the workflow
resource "aws_cloudwatch_event_rule" "guardduty_ec2_finding" {
name = "RouteHighSeverityEC2GuardDutyFindings"
description = "Triggers the IR state machine for critical EC2 findings"
event_pattern = jsonencode({
"source": ["aws.guardduty"],
"detail-type": ["GuardDuty Finding"],
"detail": {
"severity": [7, 7.0, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 8, 8.0, 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8, 8.9],
"resource": {
"resourceType": ["Instance"]
}
}
})
}
# 4. The EventBridge Target linking the Rule to the State Machine
resource "aws_cloudwatch_event_target" "step_function_target" {
rule = aws_cloudwatch_event_rule.guardduty_ec2_finding.name
target_id = "TriggerIncidentResponseStateMachine"
arn = aws_sfn_state_machine.incident_response.id
role_arn = aws_iam_role.eventbridge_to_sfn_role.arn
}
# (An IAM role for EventBridge to start the Step Function is also required)
Troubleshooting Automated Response Workflows
Automated systems can fail in complex ways. A systematic debugging approach is crucial.
❌ Cross-Account Remediation Fails
- Symptom: A Lambda in the central security account fails with `AccessDenied` when trying to modify a resource (e.g., an EC2 instance) in a member account.
- Cause: The Lambda's execution role in the security account is missing permissions to assume the required remediation role in the target member account.
- Solution:
- Ensure a remediation role (e.g., `SOAR-RemediationRole`) exists in each member account with the necessary permissions (e.g., `ec2:ModifyInstanceAttribute`).
- This member account role must have a trust policy that allows the security account's Lambda execution role to assume it.
- The security account's Lambda execution role must have `sts:AssumeRole` permission on `arn:aws:iam::MEMBER_ACCOUNT_ID:role/SOAR-RemediationRole`.
⚙️ Step Function Execution Fails on a Task
- Symptom: The Step Functions execution graph shows a specific state in red (failed).
- Cause: The Lambda function for that state failed. The state machine itself doesn't know why.
- **Solution:** Go to the failed execution in the Step Functions console. Click on the failed state. The "Execution details" will show the error output, including a link to the specific CloudWatch Log stream for that failed Lambda invocation. The logs will contain the exact exception and stack trace.
🔑 Expert-Level SecOps Best Practices
- Orchestrate with Step Functions: For any multi-step response, use Step Functions instead of a single large Lambda. It provides resilience, visibility, and error handling.
- **Use Tags for Context: Ensure all resources are tagged. Your response playbooks should use these tags to determine the business impact of a finding and adjust the response (e.g., page an engineer for `Production` resources, only email for `Development`).
- **Investigate with Detective: Don't just remediate; investigate. Use the "Investigate" link to pivot from a GuardDuty finding into Amazon Detective to understand the root cause and full scope of an incident.
- **Tune Your Alerts: Not all findings are created equal. Use GuardDuty suppression rules and Security Hub insight filters to reduce noise and focus your team's attention on the most critical threats.