Observability

const llm = new VernLLM({
  client: openai,
  model: 'gpt-4o',
  onEvent: (event) => {
    switch (event.kind) {
      case 'retry':
        metrics.increment('llm.retry', { provider: event.provider });
        break;
      case 'circuit_state':
        if (event.to === 'open') alerting.page(`circuit opened for ${event.model}`);
        break;
      case 'rate_limited':
        metrics.observe('llm.rate_limit.wait_ms', event.waitedMs, { reason: event.reason });
        break;
      case 'fallback':
        metrics.increment('llm.fallback', { from: event.from, to: event.to });
        break;
    }
  },
});

onEvent reports what happened during a call as it happens: a retry about to wait out its backoff delay, a circuit breaker changing state, an attempt that had to wait for rateLimit capacity, or the chain moving to the next fallback target. It's a single event stream instead of a separate option per concern, so handling one kind doesn't require leaving the others unset.

Fire and forget

onEvent mirrors onUsage: it is synchronous, fire and forget, and wrapped in a try/catch at the call site, so a throwing handler can never break a call.

• The handler's return value is never read. Nothing about onEvent can influence what the call does, only what gets reported about it.
• If the handler throws, the error is caught and logged through the configured logger, and the call proceeds exactly as if onEvent had not been set.
• Not configuring onEvent at all is a no-op: zero behavioral change from every version before this option existed.

const llm = new VernLLM({
  client: openai,
  model: 'gpt-4o',
  onEvent: () => {
    throw new Error('boom'); // caught and logged, never surfaces to the caller
  },
});

await llm.call({ userContent: '...' }); // still resolves normally

Event kinds

Every event kind but fallback carries provider, the name resolved for the target involved (default 'primary' for a lone target), so a single handler shared across multiple VernLLM instances, or across the targets in one instance's fallback chain, can tell them apart. fallback carries from/to instead, since it's reporting a transition between two targets rather than something that happened to one. See Provider identity.

retry

Emitted immediately before VernLLM waits for the retry backoff delay, one event per retry attempt.

{
  kind: 'retry',
  requestId: string,
  provider: string,
  model: string,
  attempt: number,
  maxRetries: number,
  delayMs: number,
  retryAfterHonored: boolean,
  error: LLMError,
}

attempt is the retry ordinal starting at 1, the first retry is 1, not the overall attempt count including the original try. model is the model actually resolved for the call, honoring a per call model override rather than always the instance default. retryAfterHonored is true when delayMs came from a Retry-After header on the failed attempt rather than computed exponential backoff. See Retries for the backoff and Retry-After mechanics this event is reporting on.

circuit_state

Emitted after a real circuit breaker state transition. No event fires for a transition that doesn't actually change anything, such as open staying open.

{
  kind: 'circuit_state',
  provider: string,
  model: string,
  from: 'closed' | 'open' | 'half-open',
  to: 'closed' | 'open' | 'half-open',
  consecutiveFailures: number,
}

provider identifies the target whose circuit changed. For a fallback chain, this lets a shared onEvent handler distinguish transitions on the primary from transitions on each fallback target. model identifies the model associated with the transition.

The meaning of consecutiveFailures and the effect of isolateByModel are covered in Circuit Breaker.

onEvent: (event) => {
  if (event.kind === 'circuit_state' && event.to === 'open') {
    alerting.page(
      `circuit opened for ${event.provider}/${event.model} after ${event.consecutiveFailures} failures`,
    );
  }
};

rate_limited

Emitted when an attempt actually had to wait for rateLimit capacity. Not emitted when capacity was immediately available, so a well-provisioned rateLimit configuration under normal load produces no rate_limited events at all.

{
  kind: 'rate_limited',
  requestId: string,
  provider: string,
  model: string,
  waitedMs: number,
  reason: 'concurrency' | 'rpm' | 'tpm',
}

reason identifies which of the three rateLimit buckets was blocking the call just before it cleared. See Rate Limiting for the buckets themselves, and Watching queue pressure before it becomes failures for using this event as a leading indicator ahead of rate_limit_queue_full/rate_limit_queue_timeout failures.

fallback

Emitted when a target's own retries are exhausted or abandoned, fallbackOn selects 'next', and the chain advances to another target, including the very first fallback after the primary fails. The failed target can be the primary or any fallback target: from names whichever one it was. Not emitted when a lone target (no fallback configured) fails, or when fallbackOn returns 'stop', since there's nowhere for the chain to move to.

{
  kind: 'fallback',
  requestId: string,
  from: string,       // the provider name that just failed
  to: string,          // the provider name about to be tried
  fromIndex: number,   // -1 for the primary
  toIndex: number,
  error: LLMError,     // the normalized error that caused the move
  elapsedMs: number,    // time spent on `from`, including its own retries
}

onEvent: (event) => {
  if (event.kind === 'fallback') {
    metrics.increment('llm.fallback', { from: event.from, to: event.to });
  }
};

See Provider Fallback for the fallback option itself and turning this event into an alert in the companion guide.

Narrowing by kind

event.kind discriminates the union, so a handler that only cares about one kind can narrow with a single check instead of leaving the others unhandled or writing an exhaustive switch:

onEvent: (event) => {
  if (event.kind !== 'circuit_state') return;
  console.warn(`circuit ${event.from} -> ${event.to}`, { provider: event.provider });
};

What this isn't

onEvent reports what happened. It has no way to change what happens next, unlike a small number of other hooks in VernLLM that genuinely do participate in the call:

onUsage and onUsageFailure are close cousins of onEvent, fire and forget in exactly the same way, but scoped to token usage rather than call mechanics. See Usage Tracking.

Options reference

See Configuration for how onEvent sits alongside onUsage, onUsageFailure, logger, and debug.