Timeouts, rate limits, and circuit breakers

Bound every external call with a timeout, respect rate limits, and trip a circuit breaker when a dependency is failing so one sick service does not sink the whole agent.

Every call gets a timeout

An agent waiting forever on a hung dependency is an agent that has failed without knowing it. Put a timeout on every external call so a slow service becomes a handled error instead of a silent stall.

Respect rate limits

Bursts of calls hit rate limits, which reorder or drop work and change the timing the agent depends on. Back off and throttle proactively rather than hammering a limited API and hoping.

Trip a breaker on a failing dependency

When a downstream service is clearly down, continuing to call it wastes time and can make things worse. The Circuit Breaker pattern pattern stops calls to a failing dependency for a cooldown, then tests it before resuming. It keeps one broken service from cascading into total failure.

pythonimport time
from concurrent.futures import ThreadPoolExecutor

_pool = ThreadPoolExecutor()

def with_timeout(fn, *args, seconds=5):
    # Bound every external call so a hung dependency becomes a handled error.
    return _pool.submit(fn, *args).result(timeout=seconds)

class CircuitBreaker:
    def __init__(self, threshold=5, cooldown=30):
        self.threshold, self.cooldown = threshold, cooldown
        self.failures, self.opened_at = 0, None

    def call(self, fn, *args):
        if self.opened_at and time.monotonic() - self.opened_at < self.cooldown:
            raise RuntimeError("circuit open: skipping the failing dependency")
        try:
            result = with_timeout(fn, *args)
        except Exception:
            self.failures += 1
            if self.failures >= self.threshold:
                self.opened_at = time.monotonic()  # stop calling for a cooldown
            raise
        self.failures, self.opened_at = 0, None  # recovered, reset
        return result

None of this code runs against a mock, because a mock replies in a millisecond and never refuses. The timeout never trips, the backoff never fires, and the circuit breaker sits there open and untested. Testing this properly requires an environment that can add latency, return failures, and take a dependency offline on command.

Tools such as WireMock and Arga create these conditions in different ways. WireMock gives HTTP-level control over delays and errors for an isolated service, while Arga provides stateful service replicas for tests that depend on persistent state across calls.