Category
Distributed-systems reliability: outbox, inbox, locking, reconciliation and exactly-once
Reliable distributed systems are designed on the assumption that messages arrive at least once, out of order, and that every step can fail. Record the business update and the event in one transaction (outbox), give dispatch workers ownership that survives COMMIT so nothing is sent twice by accident (dispatcher), commit the receipt record and the business update together (inbox), serialize races on rows that do not exist yet with advisory locks, and converge whatever drift remains by reconciling against the source of truth. Separate exactly-once delivery, processing and effect, and use idempotency keys and retries to make the business effect happen once — that is the scope of this cluster.
10 articles in total
Foundational guide
Foundational guide (start here)
The Transactional Outbox Pattern: Make the DB Update and Event Publishing Atomic, and Cut Off Lost Events and Double Publishing
An implementation guide to the transactional outbox pattern that solves the dual-write problem of distributed systems. Write to the outbox in the same transaction as the business update, publish reliably with a relay (polling/CDC), and make downstream idempotent. We explain ordering guarantees, at-least-once, and reconciliation in real code.
Related practical articles
- 信頼性アーキテクチャ設計決済AWSPython
What Exactly-Once Actually Guarantees: Separating Delivery, Processing and Effect, and Converging External Side Effects to One with Idempotency Keys and Reconciliation
Exactly-once neither 'doesn't exist' nor 'works everywhere': its meaning depends on the boundary and on what it covers — delivery, processing or effect. This sorts out, from primary sources, what Kafka transactions, SQS FIFO and database transactions guarantee and what is not guaranteed for a business effect spanning PostgreSQL, Stripe and Lambda, then shows tested Python code that converges an external API call whose response was lost to exactly one effect, using idempotency keys and reconciliation.
20 min read - PostgreSQL信頼性アーキテクチャ設計Pythonデータベース
Running an Outbox Dispatcher Safely with Multiple Workers: Why FOR UPDATE SKIP LOCKED Isn't Enough, and Leases with Fencing Tokens
When several workers process a PostgreSQL outbox table, doing SELECT ... FOR UPDATE SKIP LOCKED, committing, and then calling an external API makes two workers process the same message. This explains why, and how to claim and finalize correctly with a lease and a fencing token (lease_version) — plus backoff, dead-lettering, per-aggregate ordering and monitoring SQL — with Python code tested concurrently against PostgreSQL 18.
20 min read - PostgreSQLデータベース信頼性Pythonアーキテクチャ設計
PostgreSQL Advisory Locks in Practice: Serializing the Check-Then-Insert Race That FOR UPDATE Can't Stop
How to serialize the TOCTOU race where 'insert if it doesn't exist' succeeds twice under concurrent requests, using PostgreSQL advisory locks. Covers the difference from FOR UPDATE, session vs transaction level, PgBouncer and RDS Proxy caveats, mapping UUIDs to 64-bit keys, lock_timeout, observing locks in pg_locks, and when a constraint is the better tool — with Python (SQLAlchemy 2.x) code tested against PostgreSQL 18.
18 min read - 決済信頼性アーキテクチャ設計PostgreSQLPython
Designing Reconciliation: Detect and Classify Drift Between Your Database and Stripe, Then Repair It Safely Through an Outbox
When missed webhooks or outages leave your database and an external service such as Stripe out of step, reconciliation detects the drift, classifies it, and sends repair commands through the canonical write path. Why 'LIMIT 200 every hour' never covers everything, a keyset cursor with a full-cycle guarantee, classification that never treats unverifiable state as healthy, rate limits and monitoring — with Python code tested against PostgreSQL 18.
20 min read - 決済信頼性PostgreSQLPythonアーキテクチャ設計
Implementing Webhook Idempotency with a Transactional Inbox: The Lost Events and Double Processing That Event-ID De-duplication Alone Won't Stop
How to make webhook handling idempotent with a transactional inbox that commits the receipt record and the business update in one PostgreSQL transaction. Why marking an event processed first loses it forever and marking it afterwards processes it twice, how to handle concurrent duplicates, out-of-order delivery and separate Event objects for the same fact, and how this compares with DynamoDB de-duplication — with Python code tested against PostgreSQL 18.
25 min read - AWSSQSサーバーレス冪等性アーキテクチャ設計
Building Idempotent Async Processing with SQS + Lambda + EventBridge: Duplicate, Ordering, and DLQ Design on the At-Least-Once Premise
An implementation guide for designing AWS serverless, event-driven async processing (SQS+Lambda+EventBridge) at production quality. Explained with real code: idempotent consumers due to at-least-once delivery, visibility timeout, DLQ and reprocessing, FIFO ordering/deduplication, and partial batch failure (ReportBatchItemFailures).
25 min read - PythonCeleryRedis非同期処理タスクキュー
Celery + Redis Production-Operations Guide — Async Task Design Faithful to the Official Docs (Idempotency, Retries, Observability)
A practical guide to designing a production-grade async task queue with Celery 5.6 + Redis. Faithful to the official docs, it explains broker/backend configuration, the visibility_timeout traps, idempotency, autoretry, Beat periodic execution, Canvas workflows, prefetch tuning, observability, and security—with real code and 'when to use / when not to use' decision axes.
23 min read - アーキテクチャ設計TypeScriptAWSサーバーレスB2B SaaS
Designing Systems That Don't Fall Over When External Dependencies Do: A Retry, Exponential-Backoff + Jitter, and Circuit-Breaker Implementation Guide
A practical guide to building a 'system that doesn't fall over' against unreliable external APIs. From the great principle of retries (idempotency is the premise), exponential backoff + jitter, the timeout budget, the circuit breaker (closed/open/half-open), the bulkhead, to the trap of fallback—all explained in working TypeScript faithful to the AWS/Azure official docs.
28 min read - リアルタイムアーキテクチャ設計Next.jsTypeScriptパフォーマンス
Design Judgment for a Real-Time UI: Choosing WebSocket / SSE / Optimistic Update + Invalidation Correctly from the Requirements
A decision guide for choosing the implementation method of a real-time UI (WebSocket/SSE/polling/optimistic update + cache invalidation) from the requirements. The reason 'real-time ≠ WebSocket mandatory,' idempotent concurrent editing, SSE's wire format and implementation, and near-real-time design — explained with real code interweaving the MDN official spec and real-project judgments.
23 min read