Designing for Failure

In a distributed system, failure is not an anomaly—it's a certainty. Network partitions happen, servers crash, and third-party APIs experience downtime.

The Circuit Breaker Pattern

When a downstream service is struggling, continually sending it traffic will only make things worse. A circuit breaker detects failures and "trips", returning an immediate error (or fallback) instead of waiting for a timeout.

We can use libraries like Polly in .NET to handle this seamlessly.

var policy = Policy.Handle<HttpRequestException>()
    .CircuitBreakerAsync(
        exceptionsAllowedBeforeBreaking: 3,
        durationOfBreak: TimeSpan.FromSeconds(30)
    );

---