Tool Calling

Let the model request application defined tools, without VernLLM ever executing them

VernLLM supports first class tool calling on top of its existing reliability layer. Retries, validation, usage tracking, and caching all keep working the same way whether or not tools is set.

VernLLM never executes tools itself. call() returns the tool calls the model requested as structured data, and it is entirely up to your application to decide how and when to run them.

Tool calling introduces two breaking type changes for custom LLMClient implementations and strictly typed ConversationTurn construction. Most applications are unaffected, see Migration Notes if you implement LLMClient directly or build history entries from a loosely typed role.

ToolCall.arguments is typed per tool when a tool is declared with defineTool() and an argumentsSchema, instead of unknown. See Typed arguments per tool below.

Enabling tools

Pass tools to call() as an array of ToolDefinition objects. Each one describes a capability the model may request, not the capability itself:

tool-calling-basic-usage.ts

const weatherTool = {
  name: 'get_weather',
  description: 'Gets the current weather for a city',
  parameters: {
    type: 'object',
    properties: {
      city: { type: 'string' },
    },
    required: ['city'],
  },
};

const result = await llm.call({
  userContent: 'What is the weather in New York?',
  tools: [weatherTool],
});

Setting tools changes what call() returns. Instead of T directly, you get back a CallWithToolsResult<T> discriminated union. This is intentional and does not affect calls that omit tools, which keep returning T exactly as before.

The result shape

CallWithToolsResult<T> is either a content result or a tool call result:

tool-calling-result-types.ts

type CallWithToolsResult<T, Tools extends readonly ToolDefinition[] = ToolDefinition[]> =
  ContentResult<T> | ToolCallResult<Tools>;

type ContentResult<T> = {
  type: 'content';
  content: T;
};

type ToolCallResult<Tools extends readonly ToolDefinition[] = ToolDefinition[]> = {
  type: 'tool_calls';
  toolCalls: ToolCall<Tools>[];
  content?: string; // any text the model produced alongside the request
};

Tools defaults to ToolDefinition[], so ToolCall.arguments is unknown unless the tools you pass let TypeScript infer a more specific Tools, see Typed arguments per tool below.

Check result.type to branch:

tool-calling-branching.ts

const result = await llm.call({
  userContent: 'What is the weather in New York?',
  tools: [weatherTool],
});

if (result.type === 'tool_calls') {
  for (const call of result.toolCalls) {
    console.log(call.name, call.arguments);
  }
} else {
  console.log(result.content);
}

Prefer the exported isToolCallResult() helper over relying on TypeScript's static narrowing whenever result's type isn't statically CallWithToolsResult<T> alone, for example a conditional-tools call typed as the T | CallWithToolsResult<T> union, or a call built from a variable already widened to plain CallParams<T>. See the overload note below.

A note on the call() overload

TypeScript resolves call() to the tools-aware overload when tools is visible as a required, present key at the call site: an inline object literal with tools: [...], or a variable typed as CallParams<T> & { tools: ToolDefinition[] }.

If tools is present but conditional, e.g. tools: someCondition ? [weatherTool] : undefined, TypeScript resolves to a third overload that returns the honest union T | CallWithToolsResult<T> instead of either shape alone, since the real value follows whatever tools actually is at runtime. This works automatically for an inline literal or any variable that keeps tools's narrower, possibly-undefined type intact:

tool-calling-conditional-tools.ts

import { isToolCallResult } from 'vern-llm';

const tools = someCondition ? [weatherTool] : undefined;

const result = await llm.call({ userContent: 'What is the weather?', tools });
// result: T | CallWithToolsResult<T>

if (isToolCallResult(result)) {
  // handled safely
}

When jsonMode: false is explicit, conditional tools still infer the non-tool result as a string, including with stream: true and cachedCall(). No response-type generic is needed: the result is string | CallWithToolsResult<string>. The raw string branch applies when tools is undefined; when tools are present, a normal response is the wrapped { type: 'content', content: string } branch. The corresponding reusable parameter types are ConditionalStringToolCallParams, CachedConditionalStringToolCallParams, StreamConditionalStringToolCallParams, and CachedStreamConditionalStringToolCallParams.

This only works when the object passed to call() still carries tools's real, narrower type. If you build params as a variable explicitly annotated CallParams<T> first, that annotation widens tools away before it ever reaches call(), TypeScript has already discarded the information that would let it pick the union overload, and call() falls back to the plain Promise<T> overload regardless of what tools turns out to be at runtime. The return value's actual shape still follows tools at runtime, so this mismatch is silent rather than a compile error.

Prefer passing the object literal straight to call() (as in the example above), rather than building it up in a separately annotated variable first. If you do need a named variable, for example to reuse it across call() and cachedCall(), or to build it up conditionally before the call, use defineCallParams() instead of a : type annotation. It's an identity function that hands back exactly what you pass it, preserving tools's real, possibly-undefined shape instead of widening it the way : would, so the union overload still resolves correctly:

tool-calling-define-call-params.ts

import { defineCallParams, isToolCallResult } from 'vern-llm';

const params = defineCallParams({
  userContent: 'What is the weather?',
  tools: someCondition ? [weatherTool] : undefined,
});

const result = await llm.call(params);
// result: unknown | CallWithToolsResult<unknown> (T defaults to unknown,
// same as if params had been passed to call() inline)

if (isToolCallResult(result)) {
  // handled safely
}

Pin the content type by passing an explicit type argument to call<T>() itself, the same as you would on a plain call() without a schema (defineCallParams() has one type parameter, the whole params object's own type, not T, so T is pinned at the call() site, not on defineCallParams()):

tool-calling-define-call-params-typed.ts

const params = defineCallParams({
  userContent: 'What is the weather?',
  tools: someCondition ? [weatherTool] : undefined,
});

const result = await llm.call<string>(params);
// result: string | CallWithToolsResult<string>

defineCachedCallParams() is the same idea for cachedCall(), preserving the whole { cacheKey, ttl, call } object, call.tools included, in one reusable named variable. T defaults to unknown unless pinned via call<T>() or a schema, same as defineCallParams:

tool-calling-define-cached-call-params.ts

import { defineCachedCallParams } from 'vern-llm';

const params = defineCachedCallParams({
  cacheKey: 'weather-ny',
  ttl: 60,
  call: {
    userContent: 'What is the weather?',
    tools: someCondition ? [weatherTool] : undefined,
    jsonMode: true,
    schema: { safeParse: (d: unknown) => ({ success: true as const, data: String(d) }) },
  },
});

const result = await llm.cachedCall(params);
// result: string | CallWithToolsResult<string>

satisfies CallParams<T> also works as a lighter-weight alternative that needs no extra import, it checks the object against CallParams<T> without replacing its inferred type. The trade-off: satisfies alone doesn't pin call()'s generic T either, so it also needs an explicit call<T>() to get anything more precise than unknown, same as defineCallParams. defineCallParams/defineCachedCallParams exist mainly for the self-documenting name and to avoid re-typing satisfies CallParams<T> at every call site.

If you're already committed to a : CallParams<T> annotation for another reason, isToolCallResult() is still the safe way to handle the resulting Promise<T> mistype:

tool-calling-runtime-check.ts

import { isToolCallResult } from 'vern-llm';

const params: CallParams<string> = {
  userContent: 'What is the weather?',
  tools: someCondition ? [weatherTool] : undefined,
};

const result = await llm.call(params);

if (isToolCallResult(result)) {
  // handled safely regardless of what TypeScript inferred
}

Argument parsing and validation

Tool call arguments arrive from the provider as a JSON string on the wire. VernLLM parses them into ToolCall.arguments before returning them to you.

Add argumentsSchema to a ToolDefinition for client side validation, using the same safeParse compatible shape as the top level schema option:

tool-calling-arguments-schema.ts

import { z } from 'zod';

const weatherTool = {
  name: 'get_weather',
  description: 'Gets the current weather for a city',
  parameters: {
    type: 'object',
    properties: { city: { type: 'string' } },
    required: ['city'],
  },
  argumentsSchema: z.object({ city: z.string() }),
};

If validation fails, call() throws LLMError('validation') before returning a result. If the model requests a tool name that was not in the tools array, call() throws LLMError('validation') with code: 'unknown_tool'.

Typed arguments per tool

ToolDefinition is generic over the tool's name and its argumentsSchema's inferred argument type. When call()/cachedCall() can see the exact tools you passed, ToolCall.arguments is typed accordingly instead of unknown, with no cast or re-parse needed at the call site.

Wrap each tool in defineTool() to preserve its literal name, which is what lets TypeScript match a ToolCall back to the tool that produced it:

tool-calling-define-tool.ts

import { z } from 'zod';
import { defineTool } from 'vern-llm';

const weatherTool = defineTool({
  name: 'get_weather',
  description: 'Gets the current weather for a city',
  parameters: {
    type: 'object',
    properties: { city: { type: 'string' } },
    required: ['city'],
  },
  argumentsSchema: z.object({ city: z.string() }),
});

const result = await llm.call({
  userContent: 'What is the weather in New York?',
  tools: [weatherTool],
});

if (result.type === 'tool_calls') {
  const call = result.toolCalls[0];
  call.arguments.city; // typed as string, no cast, no re-parse
}

With a single tool in the array, arguments resolves directly, there is nothing to discriminate against. With more than one tool, arguments is a union keyed by name, and TypeScript requires narrowing on call.name before letting you read a tool specific field:

tool-calling-multiple-typed-tools.ts

const cancelOrder = defineTool({
  name: 'cancel_order',
  description: 'Cancels an order',
  parameters: {
    type: 'object',
    properties: { orderId: { type: 'string' } },
    required: ['orderId'],
  },
  argumentsSchema: z.object({ orderId: z.string() }),
});

const result = await llm.call({
  userContent: 'Cancel order #4821.',
  tools: [weatherTool, cancelOrder],
});

if (result.type === 'tool_calls') {
  for (const call of result.toolCalls) {
    if (call.name === 'get_weather') {
      call.arguments.city; // string
    } else if (call.name === 'cancel_order') {
      call.arguments.orderId; // string
    }
  }
}

A plain object literal without defineTool() still works at runtime, and single-tool narrowing still works too, since there's nothing to discriminate against. But name widens to string without defineTool() (or an explicit as const), which silently defeats narrowing the moment a second tool is added to the same tools: [...] array, call.name === 'get_weather' would no longer narrow arguments. Wrap tools in defineTool() to avoid this.

This narrowing depends on TypeScript seeing the exact tool objects at the call()/cachedCall() site. A plain const tools = [weatherTool, cancelOrder] variable (assigned once, not conditionally, and without its own type annotation) still narrows correctly when passed through, same as an inline array literal. Conditional tools (tools: someCondition ? [weatherTool] : undefined) also narrow correctly, isToolCallResult() infers Tools from the result automatically in this case, see Narrowing a conditional-tools result below.

ToolCall.arguments falls back to unknown in three cases, all variations on the same theme: TypeScript no longer has the literal tool objects to work with by the time call()/cachedCall() sees them.

The tools variable itself is explicitly annotated, e.g. const tools: ToolDefinition[] = [weatherTool, cancelOrder]. The annotation replaces the inferred literal type the same way a : CallParams<T> annotation on the whole params object does below.
The params object carries an explicit : CallParams<T> annotation, the same const-literal requirement described in A note on the call() overload below.
T is pinned explicitly (call<string>(...), cachedCall<string>(...)) alongside a literal tools array. TypeScript's own generic inference rules suppress inference for every subsequent type parameter once any leading one is explicit, Tools included, regardless of its const modifier, this is true of any TypeScript generic call with a partial explicit type argument list, not something specific to this library. Pass Tools explicitly too in that case (call<string, typeof myTools>(...)), or prefer inferring T from schema/jsonSchema instead, which doesn't touch the type argument list.

Passing Tools explicitly, whichever of the three cases above applies, still requires the tools themselves to have been declared without their own widening annotation upstream, defineTool() or a literal array works, but a variable already typed ToolDefinition[] has nothing left to recover.

Narrowing a conditional-tools result

isToolCallResult() infers Tools automatically from result's own static type whenever call()/cachedCall()'s overload resolution already produced it, which includes the conditional case, no type argument needed:

tool-calling-conditional-typed.ts

const tools = someCondition ? [weatherTool, cancelOrder] : undefined;

const result = await llm.call({
  userContent: 'What is the weather?',
  tools,
  jsonMode: true,
  schema: stringSchema,
});

if (isToolCallResult(result)) {
  const call = result.toolCalls[0];
  if (call.name === 'get_weather') {
    call.arguments.city; // typed, inferred automatically
  }
}

This only works when result's static type still carries Tools, which requires TypeScript's overload resolution to have succeeded (see A note on the call() overload). If params was instead built as a variable explicitly annotated : CallParams<T>, that annotation widens tools away before call() ever sees it, and there is nothing left for isToolCallResult() to infer from. Pass Tools explicitly as its first type argument in that case:

tool-calling-conditional-explicit-tools.ts

const params: CallParams<string> = {
  userContent: 'What is the weather?',
  tools,
  jsonMode: true,
  schema: stringSchema,
};
const result = await llm.call(params);

if (isToolCallResult<typeof tools>(result)) {
  // arguments typed per tool via the explicit override
}

The same override is needed when T is pinned explicitly instead, even with a literal tools array and no : CallParams<T> annotation anywhere:

tool-calling-pinned-t-explicit-tools.ts

const myTools = [weatherTool, cancelOrder];

// `call<string>(...)` alone loses `Tools` inference here, same underlying
// cause as the `: CallParams<T>` case above.
const result = await llm.call<string, typeof myTools>({
  userContent: 'What is the weather?',
  tools: myTools,
  jsonMode: true,
  schema: stringSchema,
});

if (isToolCallResult(result)) {
  // arguments typed per tool, inferred normally here since `result`'s
  // static type already carries `Tools` from the explicit second
  // type argument above
}

Prefer defineCallParams() over a : annotation to avoid needing the override at all, see A note on the call() overload.

Tool contract errors

VernLLM validates the tool calls returned by the model before returning them to the application. Two tool contract errors have specific LLMError.code values.

Code	Meaning
`unknown_tool`	The model requested a tool that was not included in the `tools` array.
`duplicate_tool_call_id`	The model returned multiple tool calls with the same call ID.

An unknown_tool error is raised when the model requests a tool that was not offered in the request.

A duplicate_tool_call_id error is raised when multiple tool calls in the same response use the same call ID. Tool call IDs must be unique so that the application can associate each tool result with the corresponding model request.

Both errors are surfaced as LLMError('validation') with the corresponding code:

tool-contract-errors.ts

try {
  await llm.call({
    userContent: 'Use the available tools.',
    tools: [weatherTool],
  });
} catch (error) {
  if (isLLMError(error)) {
    if (error.code === 'unknown_tool') {
      console.log('The model requested a tool that was not offered.');
    }

    if (error.code === 'duplicate_tool_call_id') {
      console.log('The model returned a duplicate tool call ID.');
    }
  }
}

A single model response can contain multiple tool contract problems. VernLLM reports the detected problems together through LLMError.issues so applications can inspect the complete set instead of handling only the first problem encountered. hasIssues narrows issues to ToolIssue[] for either tool-contract code, no manual cast needed. See issues for the full set of codes that carry typed issues.

tool-contract-issues.ts

import { isLLMError, hasIssues } from 'vern-llm';

try {
  await llm.call({
    userContent: 'Use the available tools.',
    tools: [weatherTool],
  });
} catch (error) {
  if (
    isLLMError(error) &&
    (hasIssues(error, 'unknown_tool') || hasIssues(error, 'duplicate_tool_call_id'))
  ) {
    for (const issue of error.issues) {
      console.log(issue);
    }
  }
}

Tool contract errors are not retried. An unknown_tool error cannot be repaired by retrying the same request because the available tool definitions have not changed. A duplicate_tool_call_id error is likewise a defect in the model response and retrying the same request does not change the tool contract.

These errors also do not count toward the circuit breaker. They indicate a model response problem, not provider or transport health. A provider circuit therefore cannot be opened by unknown_tool or duplicate_tool_call_id.

See Error Handling for the complete retry behavior and Circuit Breaker for how tool contract errors interact with provider health tracking.

Continuing after a tool call

VernLLM does not run tools automatically. Once your application has executed the requested tool, call call() again with the tool call and its result appended to history:

tool-calling-continuation.ts

const first = await llm.call({
  userContent: 'What is the weather in New York?',
  tools: [weatherTool],
});

if (first.type === 'tool_calls') {
  const call = first.toolCalls[0];
  const weather = await getWeather(call.arguments.city); // typed as string when weatherTool used defineTool()

  const final = await llm.call({
    userContent: 'What is the weather in New York?',
    tools: [weatherTool],
    history: [
      {
        role: 'assistant',
        toolCalls: first.toolCalls,
      },
      {
        role: 'tool',
        toolResults: [
          {
            toolCallId: call.id,
            content: weather,
          },
        ],
      },
    ],
  });
}

See the tool execution loop guide for a complete, runnable example, and Multi-turn Conversations for how tool turns fit into history more generally.

toolChoice

toolChoice controls whether and how the model must call a tool. It defaults to 'auto' when tools is set:

'auto'

The model decides whether to call a tool. This is the default whenever tools is set.

'required'

The model must call some tool, but VernLLM does not constrain which one.

'none'

The model must not call a tool, even though tools were offered.

{ name: string }

Forces the model to call the named tool. Throws LLMError('invalid_params') with code: 'unknown_tool_choice' if the name is not present in tools.

Setting toolChoice: 'none' also narrows call()'s return type to ContentResult<T> alone, instead of the full CallWithToolsResult<T> union. Since the model is structurally barred from returning a tool_calls result in that case, TypeScript can prove result.content is exactly T. No isToolCallResult narrowing check is needed at that call site:

tool-choice-none-narrowing.ts

const result = await llm.call({
  userContent: 'Summarize using the tool result above.',
  tools: [weatherTool],
  toolChoice: 'none',
  history: [
    { role: 'assistant', toolCalls: first.toolCalls },
    { role: 'tool', toolResults },
  ],
});

// No `isToolCallResult(result)` check required. `result.content` is
// already typed as the plain response, not `T | string | undefined`.
console.log(result.content);

toolChoice: 'none' is sent on the wire like any other toolChoice value, instructing the provider not to call a tool. The type-level narrowing to ContentResult<T> builds on that: if a provider (or a custom adapter) returns tool_calls anyway despite 'none', call() treats it as an API-contract violation and throws LLMError('validation') with code: 'tool_choice_none_violated' rather than returning a shape that would contradict the type you were given.

Bedrock's Converse API has no equivalent to 'none' while tools are still offered. fromBedrock throws LLMError('invalid_params') with code: 'unsupported_capability' rather than silently falling back to 'auto', which would let the model call tools despite the caller explicitly asking it not to. Omit tools entirely for that call instead.

Combining with jsonSchema

By default, on Anthropic and Bedrock, tools and jsonSchema cannot be combined. jsonSchema is implemented internally as a forced single tool call, which collides with real, caller supplied tools occupying the same request field. On unsupported Anthropic and Bedrock models, setting both throws LLMError('invalid_params') with code: 'unsupported_capability' and issues.capability: 'tools_with_json_schema', allowing the default fallback policy to try the next target. Gemini and OpenAI compatible clients never had this restriction: Gemini builds responseSchema and tools as independent, unconditional fields, and OpenAI compatible clients pass response_format and tools straight through, so jsonSchema and tools always compose there, with no opt in needed.

Both providers also support a schema constrained output mechanism that lives in its own request field, independent of tool calling, so this restriction doesn't apply on models covered by nativeStructuredOutputModels, a caller-supplied, opt-in list. See Structured Output → Combining with tools for how to enable it.

tools and schema (client side validation, not jsonSchema) are never mutually exclusive. schema only validates parsed JSON text and never touches the request fields tool calling uses. Likewise, jsonMode: true alongside tools (a plain json_object style prompt instruction, no schema) always composes: it's a system prompt nudge on every adapter, not a request field that could collide with tools.

jsonMode also defaults to false when tools is set, since forcing a JSON response format alongside tool calling is unreliable across providers. Pass jsonMode: true explicitly if you still want JSON parsing applied to the model's text content.

Caching tool calls

cachedCall() supports tool enabled calls the same way it supports plain ones. The entire CallWithToolsResult<T> is cached, including tool_calls results, not only final answers:

tool-calling-caching.ts

const result = await llm.cachedCall({
  cacheKey: `weather:${city}`,
  ttl: 60,
  call: {
    userContent: `What is the weather in ${city}?`,
    tools: [weatherTool],
  },
});

Whether caching a tool_calls result is appropriate depends on the tool. Caching "the model decided to call get_weather" is usually fine to reuse briefly. Caching a decision made under permissions or account state that can change between calls is not. Use a short ttl, or route tool_calls results through a separate cacheKey from final answers, if that distinction matters for your tools.

Breaking changes for a small number of callers

Adding tool calling required two type level changes: ConversationTurn became a discriminated union, and LLMClient.messages widened to include tool turns. Neither affects a normal call() user. See Migration Notes for exactly what breaks and why, verified against the compiler rather than guessed at.

Provider support

Every adapter translates VernLLM's OpenAI-shaped tools, tool_choice, and tool_calls into that provider's native mechanism:

Provider	Native mechanism
OpenAI compatible	Passed through as is, VernLLM's wire format already matches
Anthropic	`tool_use` / `tool_result` content blocks
Bedrock	Converse's `toolUse` / `toolResult` content blocks
Gemini	`functionCall` / `functionResponse` parts
Custom / `fromFetch`	Whatever your provider's own wire format is; `mapRequest`/`mapResponse` translate it, see Custom Providers

Gemini 3 and later provide a native call ID for each function call. On earlier models, VernLLM's Gemini adapter synthesizes a unique ID when the same tool is called more than once in a turn, so parallel calls can still be told apart when results come back. Function responses use the original function name associated with each tool call.

Tool Calling

'auto'

'required'

'none'

{ name: string }

On this page