Timeouts, rate limits, and circuit breakers

Bound every external call with a timeout, respect rate limits, and trip a circuit breaker when a dependency is failing so one sick service does not sink the whole agent.

Every call gets a timeout

An agent waiting forever on a hung dependency is an agent that has failed without knowing it. Put a timeout on every external call so a slow service becomes a handled error instead of a silent stall.

Respect rate limits

Bursts of calls hit rate limits, which reorder or drop work and change the timing the agent depends on. Back off and throttle proactively rather than hammering a limited API and hoping.

Trip a breaker on a failing dependency

When a downstream service is clearly down, continuing to call it wastes time and can make things worse. The Circuit Breaker pattern pattern stops calls to a failing dependency for a cooldown, then tests it before resuming. It keeps one broken service from cascading into total failure.

pythonimport time
from concurrent.futures import ThreadPoolExecutor

_pool = ThreadPoolExecutor()

def with_timeout(fn, *args, seconds=5):
    # Bound every external call so a hung dependency becomes a handled error.
    return _pool.submit(fn, *args).result(timeout=seconds)

class CircuitBreaker:
    def __init__(self, threshold=5, cooldown=30):
        self.threshold, self.cooldown = threshold, cooldown
        self.failures, self.opened_at = 0, None

    def call(self, fn, *args):
        if self.opened_at and time.monotonic() - self.opened_at < self.cooldown:
            raise RuntimeError("circuit open: skipping the failing dependency")
        try:
            result = with_timeout(fn, *args)
        except Exception:
            self.failures += 1
            if self.failures >= self.threshold:
                self.opened_at = time.monotonic()  # stop calling for a cooldown
            raise
        self.failures, self.opened_at = 0, None  # recovered, reset
        return result

None of this code runs against a mock, because a mock replies in a millisecond and never refuses. The timeout never trips, the backoff never fires, and the circuit breaker sits there open and untested. Testing this properly requires an environment that can behave badly on command. Arga Labs is one option, as you can add two seconds of latency to every call and watch whether the timeout fires at the boundary you set, or take a replica offline mid-run and confirm the breaker opens after the threshold you configured.

WireMock is another approach, particularly useful if you are working with HTTP APIs and want fine-grained control over response delays and failure injection without spinning up full service replicas. It is lighter to set up and works well for testing a single service in isolation, though it does not hold real state the way a full replica does.