Amazon S3 Security and Access Control Mastery

🔧 Default Security Features

# Check if your bucket has secure defaults enabled
aws s3api get-bucket-encryption --bucket YOUR-BUCKET-NAME
aws s3api get-public-access-block --bucket YOUR-BUCKET-NAME
aws s3api get-bucket-acl --bucket YOUR-BUCKET-NAME

🔧 1. SSE-S3 Encryption ON

All objects are automatically encrypted at rest using AES-256 with AWS-managed keys.

{
  "Rules": [
    {
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "AES256"
      },
      "BucketKeyEnabled": true
    }
  ]
}

🔧 2. Block Public Access ON

Prevents buckets and objects from being accidentally made public through ACLs or bucket policies.

aws s3api put-public-access-block \
  --bucket YOUR-BUCKET-NAME \
  --public-access-block-configuration \
  "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

🔧 3. ACLs Disabled

Forces all access control through IAM policies and bucket policies, eliminating ACL complexity.

aws s3api put-bucket-ownership-controls \
  --bucket YOUR-BUCKET-NAME \
  --ownership-controls Rules='[{ObjectOwnership=BucketOwnerEnforced}]'

🔧 Organization and Network Perimeter

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyOutsideOrganization",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::EXAMPLE-BUCKET",
        "arn:aws:s3:::EXAMPLE-BUCKET/*"
      ],
      "Condition": {
        "StringNotEquals": {
          "aws:PrincipalOrgID": "o-12345678"
        },
        "BoolIfExists": {
          "aws:PrincipalIsAWSService": "false"
        }
      }
    },
    {
      "Sid": "DenyUntrustedNetworks",
      "Effect": "Deny", 
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::EXAMPLE-BUCKET",
        "arn:aws:s3:::EXAMPLE-BUCKET/*"
      ],
      "Condition": {
        "StringNotEqualsIfExists": {
          "aws:SourceVpc": ["vpc-11112222", "vpc-33334444"]
        },
        "NotIpAddress": {
          "aws:SourceIp": ["203.0.113.0/24", "198.51.100.0/24"]
        },
        "BoolIfExists": {
          "aws:ViaAWSService": "false"
        }
      }
    }
  ]
}

🔧 Resource Control Policies (RCPs)

Instead of copying perimeter policies to thousands of buckets, use the new Resource Control Policy feature to enforce them organization-wide from a single location.

# Create RCP at organization root
aws organizations create-policy \
  --name "S3DataPerimeterPolicy" \
  --description "Enforce data perimeter across all S3 buckets" \
  --type RESOURCE_CONTROL_POLICY \
  --content file://data-perimeter-policy.json

# Attach to organization root or specific OUs
aws organizations attach-policy \
  --policy-id p-12345678 \
  --target-id r-abcd \
  --target-type ROOT

🔧 1. Plain Bucket Policy

When to use: Few datasets, simple role layout, traditional IAM-centric approach.

Hard limit: 20 KB policy size constrains you to tens of prefixes before hitting the limit. ≈30 prefixes max

{
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"AWS": "arn:aws:iam::123456789012:role/DataScienceRole"},
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::data-lake/science/*"
    },
    {
      "Effect": "Allow", 
      "Principal": {"AWS": "arn:aws:iam::123456789012:role/AnalyticsRole"},
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::data-lake/analytics/*"
    }
  ]
}

🔧 2. Bucket-per-Dataset

When to use: You can physically split data and create new buckets for logical separation.

Gotchas: Requires data migration and application URL rewiring when implementing. 1M buckets/account

resource "aws_s3_bucket" "dataset_buckets" {
  for_each = var.datasets
  
  bucket = "${var.org_prefix}-${each.key}-data"
}

resource "aws_iam_policy" "dataset_access" {
  for_each = var.datasets
  
  name = "${each.key}DataAccess"
  policy = jsonencode({
    Statement = [{
      Effect = "Allow"
      Action = ["s3:GetObject", "s3:ListBucket"]
      Resource = [
        aws_s3_bucket.dataset_buckets[each.key].arn,
        "${aws_s3_bucket.dataset_buckets[each.key].arn}/*"
      ]
    }]
  })
}

🔧 3. S3 Access Points

When to use: Need per-prefix policies without migrating data. Each team gets their own policy.

Gotchas: Applications must switch to Access Point endpoints instead of bucket endpoints. 10K APs/bucket

# Create Access Point
aws s3control create-access-point \
  --account-id 123456789012 \
  --name analytics-team-ap \
  --bucket data-lake \
  --policy file://analytics-team-policy.json

# Access Point Policy
{
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"AWS": "arn:aws:iam::123456789012:role/AnalyticsRole"},
    "Action": ["s3:GetObject", "s3:ListBucket"],
    "Resource": [
      "arn:aws:s3:us-east-1:123456789012:accesspoint/analytics-team-ap",
      "arn:aws:s3:us-east-1:123456789012:accesspoint/analytics-team-ap/object/analytics/*"
    ]
  }]
}

🔧 4. S3 Access Grants

When to use: Large-scale data lakes with thousands of users requiring user-level audit trails.

Best for: Directory groups mapping to datasets, human-level permissions with Identity Center integration. 100K grants/org

# Create Access Grants instance
aws s3control create-access-grants-instance \
  --account-id 123456789012 \
  --identity-center-arn arn:aws:sso:::instance/ssoins-1234567890abcdef

# Create grant for a group
aws s3control create-access-grant \
  --account-id 123456789012 \
  --access-grants-location-id default \
  --grantee Type=GROUP,Identifier=DataScientists \
  --permission READ \
  --application-arn arn:aws:s3:::data-lake \
  --s3-prefix-type Object \
  --configuration S3SubPrefix=science/

🔧 Lake Formation for Structured Data

AWS Lake Formation extends the Access Grants concept to structured data in the Glue Data Catalog, enabling table-level, column-level, and row-level security.

import boto3

lf_client = boto3.client('lakeformation')

# Grant table access with column filtering
response = lf_client.grant_permissions(
    Principal={'DataLakePrincipalIdentifier': 'arn:aws:iam::123456789012:role/AnalystRole'},
    Resource={
        'Table': {
            'DatabaseName': 'sales_data',
            'Name': 'customer_transactions',
            'ColumnNames': ['transaction_id', 'amount', 'date'],  # Column-level security
            'ColumnWildcard': {'ExcludedColumnNames': ['ssn', 'credit_card']}  # Exclude sensitive columns
        }
    },
    Permissions=['SELECT'],
    PermissionsWithGrantOption=[]
)

# Row-level security with data filters
response = lf_client.create_data_cells_filter(
    TableData={
        'DatabaseName': 'sales_data',
        'TableName': 'customer_transactions',
        'Name': 'region_filter',
        'RowFilter': {
            'FilterExpression': 'region = \'us-west-2\''  # Only show transactions from specific region
        }
    }
)

🔧 Enhanced Denial Reasons

Access-denied errors now include specific information about which policy type caused the denial and whether it was explicit or implicit.

{
  "errorCode": "AccessDenied",
  "errorMessage": "Access Denied",
  "additionalEventData": {
    "AuthorizationFailureReason": "EXPLICIT_DENY",
    "PolicyType": "BUCKET_POLICY",
    "PolicyArn": "arn:aws:s3:::my-bucket",
    "DenyingStatement": "DataPerimeterDeny"
  }
}

Resolution: Check the bucket policy for an explicit deny statement with Sid "DataPerimeterDeny" that's blocking the request.

🔧 KMS Denial Example

{
  "errorCode": "AccessDenied",
  "errorMessage": "The ciphertext refers to a customer master key that does not exist",
  "additionalEventData": {
    "AuthorizationFailureReason": "EXPLICIT_DENY", 
    "PolicyType": "KMS_KEY_POLICY",
    "KeyId": "arn:aws:kms:us-east-1:123456789012:key/01234567-89ab-cdef-0123-456789abcdef"
  }
}

Resolution: The KMS key policy doesn't grant the required permissions. Add kms:Decrypt to the key policy or use the ViaService condition.

🔧 IAM Access Analyzer for S3

The new S3-specific dashboard in IAM Access Analyzer provides a region-wide view of all buckets that are public or shared outside your organization.

# Create Access Analyzer for your organization
aws accessanalyzer create-analyzer \
  --analyzer-name S3SecurityAnalyzer \
  --type ORGANIZATION \
  --tags Key=Purpose,Value=S3SecurityAudit

# Generate findings for S3 resources
aws accessanalyzer start-resource-scan \
  --analyzer-arn arn:aws:access-analyzer:us-east-1:123456789012:analyzer/S3SecurityAnalyzer \
  --resource-arn arn:aws:s3:::my-bucket

# Review findings
aws accessanalyzer list-findings \
  --analyzer-arn arn:aws:access-analyzer:us-east-1:123456789012:analyzer/S3SecurityAnalyzer \
  --filter criteria=resourceType,equals=S3Bucket

🔧 Mixing SSE-KMS Keys

Problem: Objects encrypted with aws/s3 managed key, but bucket default changed to customer-managed CMK.

Solution: Use S3 Batch Operations with copy-in-place to re-encrypt existing objects with the new key.

aws s3control create-job \
  --account-id 123456789012 \
  --operation S3PutObjectCopy \
  --manifest 'S3ObjectManifest={Bucket=manifest-bucket,Key=manifest.csv}' \
  --priority 10 \
  --role-arn arn:aws:iam::123456789012:role/S3BatchOperationsRole

🔧 Incomplete SSE-KMS Permissions

Problem: Granted s3:PutObject but forgot kms:GenerateDataKey, causing writes to fail.

Solution: Always pair S3 write permissions with corresponding KMS permissions.

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:PutObjectAcl"],
      "Resource": "arn:aws:s3:::bucket/*"
    },
    {
      "Effect": "Allow", 
      "Action": ["kms:GenerateDataKey", "kms:GenerateDataKeyWithoutPlaintext"],
      "Resource": "arn:aws:kms:region:account:key/key-id"
    }
  ]
}

🔧 Race Conditions in Bucket Policies

Problem: Multiple teams editing the same bucket policy leads to JSON bloat and conflicting changes.

Solution: Move to S3 Access Points so each team manages their own policy document independently.