DynamoDB Single-Table Design: Patterns for High-Throughput Go Services
DynamoDB looks simple until you design your first table wrong and spend a week refactoring. The first time I used DynamoDB, I modelled it like a relational database — one table per entity type, with natural primary keys. It worked fine at low traffic, then became a mess of expensive scans as usage grew. Learning to think in DynamoDB's model — access patterns first, everything else second — was one of the more valuable architectural shifts in my career.
The Fundamental Mental Shift
In relational databases you normalise first and query later. SQL's query planner can handle most access patterns efficiently as long as you have reasonable indexes. In DynamoDB, there is no query planner. Every query you want to make must be anticipated in the key design. Design for access patterns first; everything else is secondary.
Before touching the DynamoDB console, write down every query your application needs to make. For our Amazon Ads management platform, this looked like:
- Get all campaigns for an advertiser profile
- Get all ad groups for a campaign
- Get campaign performance data for a date range
- Get all active campaigns across all advertiser profiles (for a scheduled job)
- Find campaigns by name substring (full-text — DynamoDB cannot do this, use a different store)
Identify which queries are primary (must be fast, will run frequently) and which are secondary. Let primary queries drive the key design.
Single-Table Design Fundamentals
Single-table design puts multiple entity types in one table. The benefit: related entities co-located in the same partition can be fetched in a single query. The trade-off: the key design is more abstract and harder to reason about initially.
The base pattern uses overloaded partition and sort keys:
// Table: ads-platform
// PK (partition key): entity type + ID
// SK (sort key): entity subtype + ID or metadata
// Advertiser profile record
PK: "PROFILE#profile-us-east-1-123" SK: "METADATA"
// → get a single profile by ID: GetItem(PK, SK="METADATA")
// Campaign under that profile
PK: "PROFILE#profile-us-east-1-123" SK: "CAMPAIGN#campaign-456"
// → list all campaigns for profile: Query(PK, SK begins_with "CAMPAIGN#")
// Ad group under a campaign
PK: "CAMPAIGN#campaign-456" SK: "ADGROUP#adgroup-789"
// → list all ad groups for campaign: Query(PK, SK begins_with "ADGROUP#")
// Performance snapshot (date-sorted)
PK: "CAMPAIGN#campaign-456" SK: "PERF#2026-01-14"
// → performance for date range: Query(PK, SK between "PERF#2026-01-01" and "PERF#2026-01-14")Go Implementation
const (
pkPrefix_Profile = "PROFILE#"
pkPrefix_Campaign = "CAMPAIGN#"
skPrefix_Metadata = "METADATA"
skPrefix_Campaign = "CAMPAIGN#"
skPrefix_AdGroup = "ADGROUP#"
skPrefix_Perf = "PERF#"
)
func profilePK(profileID string) string { return pkPrefix_Profile + profileID }
func campaignSK(campaignID string) string { return skPrefix_Campaign + campaignID }
// List all campaigns for a profile — single Query call
func (r *Repository) ListCampaigns(ctx context.Context, profileID string) ([]Campaign, error) {
result, err := r.dynamo.Query(ctx, &dynamodb.QueryInput{
TableName: aws.String(r.table),
KeyConditionExpression: aws.String("PK = :pk AND begins_with(SK, :prefix)"),
ExpressionAttributeValues: map[string]dynamotypes.AttributeValue{
":pk": &dynamotypes.AttributeValueMemberS{Value: profilePK(profileID)},
":prefix": &dynamotypes.AttributeValueMemberS{Value: skPrefix_Campaign},
},
})
if err != nil { return nil, err }
campaigns := make([]Campaign, 0, len(result.Items))
for _, item := range result.Items {
var c Campaign
attributevalue.UnmarshalMap(item, &c)
campaigns = append(campaigns, c)
}
return campaigns, nil
}GSI Pattern for Reverse Lookups
Single-table design works great when you query by the primary key. But sometimes you need to query by a different attribute. GSIs (Global Secondary Indexes) let you define alternate key structures for the same data.
Our convention: use GSI1PK and GSI1SK as generic overflow keys — the same overloading pattern as the main table, but for a different access dimension:
// Main table key: campaign → get by campaign ID
// GSI1: profile + status → list all enabled campaigns for a profile across all ad types
// Item structure (every campaign item includes both key sets)
type CampaignItem struct {
PK string `dynamodbav:"PK"` // "CAMPAIGN#campaign-456"
SK string `dynamodbav:"SK"` // "METADATA"
GSI1PK string `dynamodbav:"GSI1PK"` // "PROFILE#profile-123"
GSI1SK string `dynamodbav:"GSI1SK"` // "STATUS#ENABLED#CAMPAIGN#campaign-456"
// ... campaign fields
}
// Query using GSI1: all enabled campaigns for a profile
result, _ := r.dynamo.Query(ctx, &dynamodb.QueryInput{
TableName: aws.String(r.table),
IndexName: aws.String("GSI1"),
KeyConditionExpression: aws.String("GSI1PK = :pk AND begins_with(GSI1SK, :prefix)"),
ExpressionAttributeValues: map[string]dynamotypes.AttributeValue{
":pk": &dynamotypes.AttributeValueMemberS{Value: "PROFILE#profile-123"},
":prefix": &dynamotypes.AttributeValueMemberS{Value: "STATUS#ENABLED#"},
},
})TTL for Automatic Cleanup
DynamoDB TTL is one of the most underused features. Enable it on a numeric ttl attribute (Unix timestamp in seconds) and DynamoDB automatically deletes expired items within 48 hours at no additional cost.
We use it for:
- Idempotency tokens: 24-hour TTL — auto-cleanup after retry window
- Session data: 7-day TTL — no manual cleanup needed
- Processing state: 1-hour TTL — temporary markers during async jobs
- API response cache: 15-minute TTL — reduce downstream API calls
func withTTL(item map[string]dynamotypes.AttributeValue, d time.Duration) map[string]dynamotypes.AttributeValue {
item["ttl"] = &dynamotypes.AttributeValueMemberN{
Value: strconv.FormatInt(time.Now().Add(d).Unix(), 10),
}
return item
}
// Create a deduplication token that expires in 24 hours
r.dynamo.PutItem(ctx, &dynamodb.PutItemInput{
TableName: aws.String(r.table),
Item: withTTL(map[string]dynamotypes.AttributeValue{
"PK": &dynamotypes.AttributeValueMemberS{Value: "DEDUP#" + key},
"SK": &dynamotypes.AttributeValueMemberS{Value: "TOKEN"},
}, 24*time.Hour),
ConditionExpression: aws.String("attribute_not_exists(PK)"),
})Conditional Writes and Optimistic Locking
DynamoDB conditional writes let you make updates atomic without full transactions. For concurrent campaign updates, optimistic locking with a version number prevents lost updates:
func (r *Repository) UpdateCampaignBid(ctx context.Context, campaignID string, newBid float64, version int) error {
_, err := r.dynamo.UpdateItem(ctx, &dynamodb.UpdateItemInput{
TableName: aws.String(r.table),
Key: map[string]dynamotypes.AttributeValue{
"PK": &dynamotypes.AttributeValueMemberS{Value: "CAMPAIGN#" + campaignID},
"SK": &dynamotypes.AttributeValueMemberS{Value: "METADATA"},
},
UpdateExpression: aws.String("SET defaultBid = :bid, #v = :newVersion"),
ConditionExpression: aws.String("#v = :expectedVersion"),
ExpressionAttributeNames: map[string]string{"#v": "version"},
ExpressionAttributeValues: map[string]dynamotypes.AttributeValue{
":bid": &dynamotypes.AttributeValueMemberN{Value: fmt.Sprintf("%.2f", newBid)},
":newVersion": &dynamotypes.AttributeValueMemberN{Value: strconv.Itoa(version+1)},
":expectedVersion": &dynamotypes.AttributeValueMemberN{Value: strconv.Itoa(version)},
},
})
var condErr *dynamotypes.ConditionalCheckFailedException
if errors.As(err, &condErr) {
return ErrVersionConflict // caller should re-read and retry
}
return err
}Batch Operations
BatchWriteItem handles up to 25 items per call. For bulk imports (loading historical campaign data, for example), parallelise with controlled concurrency and handle unprocessed items:
func (r *Repository) BulkWrite(ctx context.Context, items []map[string]dynamotypes.AttributeValue) error {
const batchSize = 25
var eg errgroup.Group
sem := make(chan struct{}, 5) // max 5 concurrent batch calls
for i := 0; i < len(items); i += batchSize {
end := i + batchSize
if end > len(items) { end = len(items) }
batch := items[i:end]
sem <- struct{}{}
eg.Go(func() error {
defer func() { <-sem }()
return r.writeBatch(ctx, batch)
})
}
return eg.Wait()
}
func (r *Repository) writeBatch(ctx context.Context, items []map[string]dynamotypes.AttributeValue) error {
requests := make([]dynamotypes.WriteRequest, len(items))
for i, item := range items {
requests[i] = dynamotypes.WriteRequest{PutRequest: &dynamotypes.PutRequest{Item: item}}
}
result, err := r.dynamo.BatchWriteItem(ctx, &dynamodb.BatchWriteItemInput{
RequestItems: map[string][]dynamotypes.WriteRequest{r.table: requests},
})
if err != nil { return err }
// Retry unprocessed items (throttling)
if len(result.UnprocessedItems) > 0 {
time.Sleep(100 * time.Millisecond)
return r.writeBatch(ctx, extractItems(result.UnprocessedItems))
}
return nil
}When NOT to Use DynamoDB
DynamoDB is not the right tool for every problem. We use it for high-throughput key-value lookups, session data, idempotency tokens, and simple entity storage. We do not use it for: full-text search (use Elasticsearch or OpenSearch), complex aggregations (use a relational database or data warehouse), or ad-hoc analytics queries (use Redshift or Athena).
The access-pattern-first design philosophy works because DynamoDB makes a fundamental trade-off: you lose query flexibility in exchange for predictable, millisecond-latency performance at any scale. Design around that trade-off rather than against it.
Comments
Post a Comment