System Design Tradeoffs Start With Constraints, Not Technology (Turning Product Requirements Into Architecture)

"A seat must never be sold twice." That single product requirement determines more of a ticketing system's architecture than the choice between PostgreSQL and DynamoDB, Kafka and RabbitMQ, or monolith and microservices. Yet most design discussions begin with the stack and work backward to the requirements.

That ordering is the root of many architectural mistakes. A technology choice is an answer. Requirements are the question. When the answer comes first, the system ends up shaped by what the tools do well rather than by what the product needs to guarantee.

A more reliable approach treats system design as constraint solving. Functional requirements define invariants. Non-functional requirements define bounds on latency, availability, durability, and consistency. Scale and cost define the feasible region. Architecture is whatever simple design satisfies all of them at once, and system design tradeoffs appear exactly where constraints conflict.

This article walks through that reasoning using a hypothetical event ticketing platform. It shows how to turn vague requirements into quantified constraints, how each constraint eliminates candidate designs, where coordination must happen, and what changes as the system grows by 10x or 100x.

Why Requirements Are Constraints, Not Features

A requirement like "the checkout should be fast" is not yet a constraint. It cannot rule out any design. "Seat reservation responds within 300 ms at p99 under 2,000 requests per second" is a constraint, because it makes certain designs infeasible. A reservation path with three serial network calls to services whose individual p99 is 150 ms cannot meet it.

The first step in design is therefore translation. Each requirement becomes one of four things:

• An invariant: something that must always be true ("no seat is held by two orders").
• A bound: a quantified limit on latency, availability, durability, or staleness.
• A load profile: request rates, data volume, growth rate, and, critically, the shape of the traffic over time.
• A budget: what the organization will pay in infrastructure cost and operational complexity.

Consider what happens when a constraint is violated quietly. The ticketing platform's reservation service writes to a relational database. During a popular on-sale, thousands of users target the same few hundred seats. Row-level lock contention rises, so individual queries slow from 5 ms to 400 ms. The database is not down. It is merely slow, which is worse, because nothing fails fast.

The causal chain is predictable:

slow queries → connections held longer, so the pool exhausts → API workers block waiting for a connection → inbound requests queue in front of the workers → latency exceeds client timeouts → clients retry, often automatically at several layers → effective load multiplies on a database that is already saturated → throughput falls further

Nothing in this chain is a bug. Every component behaved as configured. The failure came from an unexamined assumption: that database latency was roughly constant. Under contention, latency is a function of load, and the system's design had no mechanism for the case where that function turned steep.

This is why constraints must be stated explicitly. "No double-selling" plus "tens of thousands of users arriving within seconds of an on-sale" together define a problem with no easy answer. Either constraint alone is straightforward. The architecture lives in their intersection.

Distributed Systems Deep Dive: How Each Constraint Eliminates Designs

Invariants Decide Where Coordination Must Happen

An invariant that spans multiple concurrent actors requires a serialization point. For "a seat is held by at most one order," some component must observe competing requests for the same seat and decide one winner. That component can be a database row with a conditional write, a single-threaded partition owner, or a consensus group, but it must exist.

The mental model is simple: every invariant has a home, and that home is a coordination bottleneck. The design question is how small the coordination scope can be made.

A conditional update keeps coordination scoped to one row:

-- Succeeds only if the seat is free or its previous hold expired.
UPDATE seats
SET status = 'held',
    hold_id = $1,
    held_until = now() + interval '10 minutes'
WHERE event_id = $2
  AND seat_id = $3
  AND (status = 'available'
       OR (status = 'held' AND held_until < now()));
-- Exactly one row affected means this request owns the seat.

This enforces the invariant at the point of write rather than at the point of read. That distinction matters: the seat map a user sees can be stale, because the authoritative check happens later and is atomic.

Scoping coordination to a seat suggests partitioning by event_id, since no invariant spans events. Partitioning increases aggregate throughput, but it does nothing for a single hot event. If one event receives 90% of traffic during its on-sale, the partition that owns it becomes the system's throughput ceiling regardless of how many partitions exist. Horizontal scaling helps workloads that spread. It does not help workloads that concentrate.

Latency Budgets and the Arithmetic of Dependencies

Latency constraints are additive along serial paths and maximal across parallel fan-out. That arithmetic eliminates designs before any code is written.

For serial calls, budgets subtract. A 300 ms p99 budget with an API gateway, an auth check, an inventory write, and a pricing lookup leaves each step a fraction of the total, and p99s do not simply add (the combination is usually better than the sum, but worst-case alignment does happen under correlated load).

Fan-out is subtler. If a request queries 100 backends in parallel and each independently exceeds its own p99 1% of the time, the probability that at least one is slow is 1 − 0.99^100, roughly 63%. The aggregate p50 inherits the components' p99. Systems that fan out widely need hedged requests, partial results, or narrower fan-out, or they must accept a latency bound far above any single component's.

Availability compounds the same way. Five serial dependencies at 99.9% availability each yield about 99.5% for the path, assuming independent failures. A product requirement of 99.9% for checkout is therefore infeasible with five hard synchronous dependencies at that same level. The constraint forces a decision: remove dependencies from the critical path, make some of them optional with degraded behavior, or relax the target.

Deadlines should travel with the request so downstream work stops when the caller has already given up:

func (h *Handler) Reserve(w http.ResponseWriter, r *http.Request) {
    // The end-to-end budget is set once at the edge.
    ctx, cancel := context.WithTimeout(r.Context(), 300*time.Millisecond)
    defer cancel()

    // Each dependency receives whatever budget remains, not a fresh timeout.
    if err := h.inventory.Hold(ctx, seatReq(r)); err != nil {
        if errors.Is(err, context.DeadlineExceeded) {
            http.Error(w, "try again", http.StatusServiceUnavailable)
            return
        }
        // ...
    }
}

Without deadline propagation, a database keeps executing queries for clients that disconnected seconds ago. That is wasted capacity at precisely the moment capacity is scarcest.

Consistency Is a Per-Operation Requirement

"Should the system be strongly or eventually consistent?" is usually the wrong question. Consistency is a property of individual operations and the decisions they support.

In the ticketing platform:

This table is effectively an architecture. The catalog can be cached aggressively and served from replicas. The seat map can be served from a cache with a short TTL. Only the hold operation requires a single authoritative writer. The orders page needs session-level consistency, which can be satisfied by routing a user's reads to the primary briefly after a write, without making the whole system strongly consistent.

The tradeoff being made here is explicit: weaker consistency on reads buys latency and throughput, and correctness is preserved because every decision that matters is checked against the authoritative store.

Durability and Cost Define the Edges of the Feasible Region

Durability is a question about when a write is acknowledged. Acknowledging after a local commit with asynchronous replication means a primary failure can lose recently confirmed orders. Acknowledging after a synchronous replica confirms adds a network round trip to every write, and cross-region synchronous replication adds tens of milliseconds or more.

Cost constrains everything else. A 99.9% monthly availability target allows about 43.8 minutes of downtime per month. A 99.99% target allows about 4.4 minutes. That additional nine typically requires multi-region failover, automated detection, and rehearsed recovery, which is a large step up in both infrastructure and operational effort. If the business impact of 40 minutes of downtime is tolerable, the cheaper design is the correct one.

Architecture: A System Shaped by Its Constraints

Applying the constraints above yields an architecture where each path is designed for its own requirements:

                Client
                  |
                  v
          Edge / Load Balancer
           |              |
   (read path)        (write path)
           |              |
           v              v
   Catalog API       Admission Control
           |         (waiting room, rate limit)
           v              |
   Cache + Read           v
   Replicas          Reservation API
                          |
                          v
                 Inventory Store (primary)
                 partitioned by event_id
                          |
                     orders + outbox
                     (same transaction)
                          |
                          v
                     Event Stream
                    /     |      \
                   v      v       v
             Payment   Email    Analytics
             Worker    Worker   Pipeline
                |
                v
          Payment Provider
          (idempotency keys)

Request flow. Reads never touch the inventory primary. Writes pass through admission control first, so the inventory store sees a bounded request rate even during an on-sale spike. Excess users wait in a queue with a visible position rather than hammering an endpoint that will time out.

State ownership. The inventory store owns seat state. No other component writes it. Caches and replicas hold derived copies whose staleness is acceptable by the table above.

Consistency boundary. The order record and an outbox row are written in the same local transaction. A relay publishes outbox rows to the event stream. This avoids a dual write, where the database commit succeeds and the publish fails (or the reverse), leaving the two systems disagreeing permanently.

Failure boundaries. If the email worker is down, orders still complete and messages accumulate in the stream. If the payment provider is slow, holds remain valid for their 10-minute window, and the payment worker retries with the same idempotency key so a retry cannot charge twice.

Scaling boundary. Read capacity scales with replicas and cache. Write capacity for a single event is bounded by one partition, which is why admission control exists: it shapes demand to fit a serialized resource that cannot be scaled out.

System Design Tradeoffs and a Decision Framework

Each choice in the architecture above solves one problem and creates another:

A useful framework for working through these decisions:

1. State the invariants first. Each one identifies a required coordination point.
2. Quantify non-functional requirements per operation, not per system. Latency, staleness, and availability usually differ by path.
3. Find the binding constraint. In most systems one constraint dominates; for this platform it is "no oversell during concentrated spikes."
4. Choose the simplest design that satisfies all constraints, including cost and team capacity.
5. Ask what breaks at 10x. Identify which constraint fails first and whether the design can evolve to meet it without a rewrite.

Note that autoscaling the stateless tier appears as a trap. Adding API instances increases concurrent database connections, which increases lock contention on hot rows. Scaling the wrong layer moves load toward the bottleneck.

Failure Modes and What Happens at Scale

The same requirements produce different architectures at different scales, and complexity is only justified when a constraint actually binds.

At 10 requests per second, a single relational database with conditional writes, synchronous payment calls, and a cron job for emails satisfies every constraint. Adding queues, partitions, and a waiting room here would be premature and would introduce failure modes the system does not need.

At 1,000 requests per second, read traffic dominates, so caching and replicas matter. Retries become frequent enough that duplicate requests are routine rather than rare, so idempotency keys on holds and payments become mandatory. Connection pool sizing becomes a capacity decision rather than a default.

At 100,000 requests per second during an on-sale, the binding constraint is per-event write serialization. Admission control and load shedding become the primary defense. Without them, the slow-dependency chain described earlier is not a risk; it is the expected outcome.

Failure modes worth designing for explicitly:

• Ambiguous outcomes. A payment call that times out may have succeeded. The system must reconcile using the idempotency key rather than assume failure.
• Duplicate messages. The outbox relay delivers at least once. The email worker must deduplicate by order ID, or customers receive duplicate confirmations.
• Reordered events. A "hold expired" event processed after an "order paid" event must not release a sold seat. Consumers should check current state or version numbers rather than trust arrival order.
• Retry storms. Retries without jitter, budgets, or backoff synchronize clients and multiply load on a recovering dependency.
• Stale reads. Acceptable on the seat map because the hold is authoritative; unacceptable if any code path makes a sale decision from cached data.

Engineering Principles for Constraint-Driven Design

1. Every invariant has a home, and that home is a coordination point. Design the scope of coordination before choosing the technology that provides it.
2. Unquantified requirements cannot constrain a design. "Fast" and "reliable" eliminate nothing; numbers do.
3. Consistency belongs to operations, not systems. Most of a system can be eventually consistent as long as every decision that protects an invariant is made against authoritative state.
4. Availability multiplies across hard dependencies. Each synchronous call on a critical path lowers the ceiling for that path.
5. Scaling compute does not solve a serialized bottleneck. When demand concentrates on one resource, shape the demand instead.
6. A timeout is part of the failure model, not a client setting. Deadlines must propagate, and timeouts must be paired with a plan for ambiguous outcomes.
7. Complexity should be purchased by a binding constraint. Each component added should name the requirement it satisfies.

Final Thoughts

The deeper idea behind constraint-driven design is that architecture is not chosen; it is derived. Once invariants are explicit and non-functional requirements are quantified, many of the apparent choices disappear. The remaining system design tradeoffs are the genuine ones, where two constraints pull in opposite directions and a decision has real consequences.

This reframing also changes how designs should be evaluated. A good architecture is not the one with the most sophisticated components. It is the one where every component can be traced back to a constraint, every coordination point protects a named invariant, and every failure mode has a defined behavior. A simple design that satisfies the requirements is superior to an elaborate one that anticipates scale that may never arrive.

These are the kinds of engineering problems that sit at the center of the work at PulseSoft: understanding where state lives, which guarantees each path actually needs, and what happens when the assumptions between components stop being true. Technology choices matter, but they are downstream of that reasoning.

Key Takeaways

• Translating requirements into invariants, quantified bounds, load profiles, and budgets is the step that makes architecture derivable instead of arbitrary.
• Enforcing invariants at the point of write, through conditional updates or single-owner partitions, allows reads to be cached and stale without compromising correctness.
• Partitioning raises aggregate throughput but cannot help a single hot key, so concentrated demand requires admission control or load shedding rather than more instances.
• Serial dependencies multiply unavailability and fan-out amplifies tail latency, so the shape of the call graph constrains achievable latency and availability targets.
• Writing state and its outgoing event in one local transaction (the outbox pattern) avoids dual-write inconsistency, at the cost of at-least-once delivery that consumers must handle idempotently.
• Each additional nine of availability carries a sharp increase in infrastructure and operational cost, so the target should reflect the actual business impact of downtime.
• Adding a component is justified only when a specific constraint binds; at low scale, the simplest design that satisfies all invariants is the correct one.