Per-call Overrides

A VernLLM instance is configured once with a default model (and other options like defaultMaxTokens), but almost every generation-related field on CallParams can be overridden for an individual call() without creating a second instance.

const llm = new VernLLM({
  client: openai,
  model: 'gpt-4o-mini', // cheap default for most calls
});

// Uses gpt-4o-mini, the instance default
const summary = await llm.call({
  userContent: 'Summarize this in one sentence.',
});

// Overrides the model for just this call
const analysis = await llm.call({
  model: 'gpt-4o',
  temperature: 0.2,
  userContent: 'Do a careful multi-step analysis of this contract.',
});

What can be overridden per call

reasoningEffort and budgetTokens each resolve through the same three layers every other per-call option does: the per-call value, then the fallback target's own default, then the instance-level default, first one set wins. See Call Params for the full conversion table between the two, and per-provider behavior.

jsonMode, jsonSchema, and schema aren't instance-level options at all — they only ever exist per call, so there's nothing to "override" at the instance level, you just set them on the calls that need them.

const llm = new VernLLM({ client: openai, model: 'gpt-4o-mini' });

await llm.call({
  systemPrompt: '...',
  userContent: '...',
  model: 'o3',
  reasoningEffort: 'high', // passed through as `reasoning_effort` for supported models
});

Common pattern: cheap default, expensive escalation

A typical setup uses a fast/cheap model by default and escalates to a stronger model only for calls that need it, rather than paying for the expensive model on every request:

async function extractCandidate(resumeText: string) {
  return llm.call({
    model: 'gpt-4o-mini',
    userContent: resumeText,
    schema: CandidateSchema,
  });
}

async function reviewFlaggedResume(resumeText: string) {
  // Harder task, worth the stronger model
  return llm.call({
    model: 'gpt-4o',
    temperature: 0.1,
    userContent: resumeText,
    schema: CandidateSchema,
  });
}

Per-user or per-tenant model selection

Because model is just a string on CallParams, it can come from anywhere at call time — a config lookup, a feature flag, a per-tenant setting:

async function callForTenant(tenantId: string, userContent: string) {
  const model = await getTenantModelPreference(tenantId); // e.g. 'gpt-4o' or 'gpt-4o-mini'

  return llm.call({ model, userContent });
}

Interaction with usage tracking

onUsage reports the model that actually served the request, including a per-call override, not the instance's default. This matters if you bill different models at different rates. The same applies to onUsageFailure, since both share the same TokenUsage shape:

const llm = new VernLLM({
  client: openai,
  model: 'gpt-4o-mini',
  onUsage: ({ model, promptTokens, completionTokens }) => {
    // `model` reflects the override, e.g. 'gpt-4o', not 'gpt-4o-mini'
    billing.record({ model, promptTokens, completionTokens });
  },
});

See Usage Tracking for the full onUsage shape.

Interaction with caching

If you cache calls that use different model or temperature overrides under the same cacheKey, you'll get a cached result from whichever call happened to populate the cache first, regardless of which override the current call is using. Fold all inputs that affect the output into the key:

const cacheKey = `analysis:${docId}:${documentHash}:${model}:${temperature}:schema-v2`;

Include a stable document revision/content hash and any generation settings that affect the output, such as model, temperature, and schema version.

See Caching for more on choosing cache keys.