Posts

Showing posts with the label Distributed Systems

Event-Driven Architecture on AWS: SQS, EventBridge and Idempotency

Image
Event-driven architecture sounds clean in diagrams: one service publishes an event, another service reacts, and the system becomes nicely decoupled. In production it is messier. Events arrive late, arrive twice, arrive out of order, or fail halfway through a workflow. AWS gives you strong building blocks, but the architecture still depends on how you handle those realities. Use EventBridge for Routing, SQS for Work The pattern I like is EventBridge for routing and SQS for durable work queues. EventBridge is good at publishing domain events and letting consumers subscribe without tight coupling. SQS is good at giving workers a queue they can drain, retry, and monitor. { "source": "ads.campaigns", "detail-type": "CampaignBudgetChanged", "detail": { "companyId": 5000, "profileId": 50000100, "campaignId": 123456789, "oldBudget": 50.00, "newBudget": 75.00 } } ...

Idempotency Keys in Distributed Go Services

Image
Distributed systems are fundamentally unreliable. Networks drop packets, services restart mid-request, clients retry on timeout, and load balancers reroute connections. The standard response to this reality is to design every state-modifying operation to be idempotent — safe to call multiple times with the same result as calling it once. Idempotency keys are the primary tool for achieving this at the API level. The Core Problem Consider a client that sends a POST request to create a campaign. The server receives the request, creates the campaign, and then a network failure prevents the response from reaching the client. The client, having received no response, retries the request. Without idempotency, the server creates a second campaign. Now you have two identical campaigns — a data integrity problem that is difficult to detect and painful to clean up. We manage campaigns for thousands of advertisers. A double-creation bug is not just a data integrity issue — it is a budget issu...