Building a Resilient K2 Service Broker

A service broker is the seam between K2 and the messy outside world. When the endpoint behind it is slow or flaky, that mess leaks straight into your workflows. Here are the patterns I reach for to keep a broker calm under pressure.

Set a timeout before anything else

The default for most HTTP clients is far too generous. A broker that hangs for a hundred seconds will exhaust the K2 worker threads long before it returns a useful error. Pick a budget and enforce it.

using var client = new HttpClient
{
    Timeout = TimeSpan.FromSeconds(15),
};

Retry, but with backoff

Transient failures deserve a second chance, not five immediate ones. Exponential backoff with a little jitter spreads the load and avoids hammering a service that is already struggling.

async Task<T> WithRetry<T>(Func<Task<T>> action, int attempts = 3)
{
    for (var i = 0; ; i++)
    {
        try { return await action(); }
        catch (Exception) when (i < attempts - 1)
        {
            var delay = TimeSpan.FromMilliseconds(200 * Math.Pow(2, i));
            await Task.Delay(delay);
        }
    }
}

Surface errors honestly

The worst broker is one that swallows a failure and returns an empty row. Map remote failures to a clear ServiceBroker error so the workflow can branch on it, and log enough context to debug later.

A broker that fails loudly and predictably is worth far more than one that looks tidy until the day it quietly loses your data.