DynamoDB Serverless Patterns

We start every DynamoDB design by listing all access patterns, then designing the key schema to support them with the minimum number of indexes.

# Access Pattern Matrix — SaaS project management app
# Pattern                        | Key Condition            | Index
# Get user by ID                 | PK=USER#id, SK=META      | Table
# List projects by user          | PK=USER#id, SK=PROJ#*    | Table
# Get project by ID              | PK=PROJ#id, SK=META      | Table
# List tasks by project          | PK=PROJ#id, SK=TASK#*    | Table
# List tasks by assignee         | GSI1PK=USER#id, GSI1SK=TASK#* | GSI1
# List tasks by due date         | GSI1PK=PROJ#id, GSI1SK=DUE#date | GSI1
# Get task by ID                 | PK=TASK#id, SK=META      | Table
# List recent activity           | GSI2PK=PROJ#id, GSI2SK=ACT#ts | GSI2

# Entity definitions
interface UserEntity {
  PK: `USER#${string}`;       // Partition key
  SK: 'META';                  // Sort key
  GSI1PK: `ORG#${string}`;    // For listing users in org
  GSI1SK: `USER#${string}`;   // Sort by user ID
  type: 'user';
  email: string;
  name: string;
  role: 'admin' | 'member';
}

interface TaskEntity {
  PK: `PROJ#${string}`;       // In project partition
  SK: `TASK#${string}`;       // Task ID as sort key
  GSI1PK: `USER#${string}`;   // Assignee lookup
  GSI1SK: `TASK#${string}`;   // Task ID
  type: 'task';
  title: string;
  status: 'todo' | 'doing' | 'done';
  assigneeId: string;
  dueDate: string;
}

The access pattern matrix is the source of truth. We never add a GSI without a documented access pattern that requires it. This prevents index bloat and keeps write costs predictable.

We use GSI overloading to serve multiple access patterns from a single index, and sparse indexes to index only a subset of items.

// GSI1 overloading — serves 3 different access patterns
// depending on the entity type stored in GSI1PK/GSI1SK

// Pattern 1: List users in org
// GSI1PK = ORG#org123, GSI1SK = USER#user456

// Pattern 2: List tasks by assignee  
// GSI1PK = USER#user456, GSI1SK = TASK#task789

// Pattern 3: List invoices by customer
// GSI1PK = CUST#cust123, GSI1SK = INV#2025-01-15

// Sparse index — only items with GSI2PK are indexed
// Use case: index only "flagged" or "urgent" items
resource "aws_dynamodb_table" "main" {
  name         = "app-${var.env}"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "PK"
  range_key    = "SK"

  attribute {
    name = "PK"
    type = "S"
  }
  attribute {
    name = "SK"
    type = "S"
  }
  attribute {
    name = "GSI1PK"
    type = "S"
  }
  attribute {
    name = "GSI1SK"
    type = "S"
  }
  attribute {
    name = "GSI2PK"
    type = "S"
  }
  attribute {
    name = "GSI2SK"
    type = "S"
  }

  global_secondary_index {
    name            = "GSI1"
    hash_key        = "GSI1PK"
    range_key       = "GSI1SK"
    projection_type = "ALL"
  }

  # Sparse index — only items with GSI2PK attribute are indexed
  global_secondary_index {
    name            = "GSI2"
    hash_key        = "GSI2PK"
    range_key       = "GSI2SK"
    projection_type = "INCLUDE"
    non_key_attributes = ["type", "title", "status", "createdAt"]
  }

  ttl {
    attribute_name = "expiresAt"
    enabled        = true
  }

  point_in_time_recovery {
    enabled = true
  }

  stream_enabled   = true
  stream_view_type = "NEW_AND_OLD_IMAGES"
}

Sparse indexes are powerful for filtering: only items that have the GSI2PK attribute appear in the index. An "urgent tasks" index costs nothing for non-urgent tasks because they are not indexed. INCLUDE projection reduces storage costs by only projecting the attributes needed for the query.

DynamoDB TTL automatically deletes expired items at no cost. We use TTL for session tokens, temporary data, audit log rotation, and soft deletes.

// TTL patterns

// 1. Session tokens — expire after 24 hours
const session = {
  PK: `SESSION#${sessionId}`,
  SK: 'META',
  userId: user.id,
  createdAt: now,
  expiresAt: Math.floor(Date.now() / 1000) + 86400,  // Unix timestamp
};

// 2. Rate limiting windows — expire after window closes
const rateLimit = {
  PK: `RATE#${ip}#${minuteBucket}`,
  SK: 'COUNT',
  count: 1,
  expiresAt: Math.floor(Date.now() / 1000) + 120,  // 2 min after window
};

// 3. Soft deletes — mark deleted, TTL after 30 days
async function softDelete(pk: string, sk: string) {
  await docClient.send(new UpdateCommand({
    TableName: TABLE,
    Key: { PK: pk, SK: sk },
    UpdateExpression: 'SET deleted = :true, expiresAt = :ttl REMOVE GSI1PK, GSI1SK',
    ExpressionAttributeValues: {
      ':true': true,
      ':ttl': Math.floor(Date.now() / 1000) + (30 * 86400),
    },
  }));
  // Removing GSI keys makes item invisible to index queries immediately
  // TTL deletes it permanently after 30 days
}

// 4. DynamoDB Streams capture TTL deletions
// event.Records[].userIdentity.type === 'Service'
// event.Records[].userIdentity.principalId === 'dynamodb.amazonaws.com'
// Use this to trigger cleanup actions (e.g., delete S3 objects)

TTL deletions appear in DynamoDB Streams with a special service principal, letting you trigger cleanup logic — like deleting associated S3 objects or sending expiration notifications — when items are automatically removed.

DynamoDB Streams captures every change to your table as an ordered sequence of events. We wire Streams to Lambda for real-time event processing, search index updates, and cross-region replication.

resource "aws_lambda_event_source_mapping" "stream" {
  event_source_arn                   = aws_dynamodb_table.main.stream_arn
  function_name                      = aws_lambda_function.stream_processor.arn
  starting_position                  = "LATEST"
  batch_size                         = 100
  maximum_batching_window_in_seconds = 5
  parallelization_factor             = 10
  maximum_retry_attempts             = 3
  bisect_batch_on_function_error     = true
  
  destination_config {
    on_failure {
      destination_arn = aws_sqs_queue.stream_dlq.arn
    }
  }
  
  filter_criteria {
    filter {
      pattern = jsonencode({
        eventName = ["INSERT", "MODIFY"]
        dynamodb = {
          NewImage = {
            type = { S = ["task"] }
          }
        }
      })
    }
  }
}

// Stream processor Lambda
export const handler = async (event: DynamoDBStreamEvent) => {
  for (const record of event.Records) {
    const newImage = unmarshall(record.dynamodb!.NewImage! as any);
    
    switch (newImage.type) {
      case 'task':
        // Update Elasticsearch index
        await esClient.index({
          index: 'tasks',
          id: newImage.SK.replace('TASK#', ''),
          body: {
            title: newImage.title,
            status: newImage.status,
            projectId: newImage.PK.replace('PROJ#', ''),
            updatedAt: newImage.updatedAt,
          },
        });
        break;
    }
  }
};

Event source mapping filters reduce Lambda invocations by filtering events at the service level — you only process the events you care about. Bisect-on-error splits a failed batch in half and retries each half independently, preventing a single bad record from blocking the entire stream. The DLQ captures records that fail after all retries for manual investigation.