Features Overview
The resilience and observability primitives VernLLM wraps around every call
Everything below is opt-in beyond sensible defaults, mix and match whichever of these your app needs.
Provider Fallback
Run declared backup targets only when the fallback policy allows the transition
Circuit Breaker
Stop hammering a provider that's down
Retries
Exponential backoff with jitter, and Retry-After awareness
Rate Limiting
Stay under a provider's requests/tokens/concurrency limits before they reject you
Usage Metering
Reserve usage before LLM calls and refund failed calls
Caching
Wrap calls with cachedCall to avoid duplicate work and reduce provider usage
Observability
One event stream for retries, circuit state transitions, rate limit waits, and fallback
Usage Tracking
Track token usage from provider responses
Error Handling
LLMError types, what triggers each, and what gets retried
Cancellation & Timeouts
How AbortSignal, per-attempt timeouts, and retries interact
Structured Output
Client-side validation with Zod and provider-native JSON Schema mode
Tool Calling
Let the model request application defined tools, without VernLLM ever executing them
Streaming
Incremental chunks as the model generates, with the same retry and caching guarantees
Pluggable Logger
Swap in your own logger implementation