Structured Extraction with Caching

Combining schema validation and cachedLLMCall for a realistic extraction pipeline

A common pattern: extract structured data from unstructured text, validate it, and cache the result so re-processing the same input does not cost another API call.

import OpenAI from 'openai';
import { VernLLM, InMemoryCacheAdapter } from 'vern-llm';
import { z } from 'zod';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const llm = new VernLLM({
  client: openai,
  model: 'gpt-4o',
  cache: new InMemoryCacheAdapter(),
});

const CandidateSchema = z.object({
  name: z.string(),
  email: z.string().email(),
  skills: z.array(z.string()),
  yearsExperience: z.number(),
});

async function parseResume(resumeId: string, resumeText: string, userId: string) {
  return llm.cachedLLMCall({
    cacheKey: `resume:${resumeId}`,
    ttl: 86_400, // 1 day
    call: {
      systemPrompt: 'Extract candidate details as JSON matching the given fields.',
      userContent: resumeText,
      schema: CandidateSchema,
    },
    reserveUsage: () => quota.reserve(userId),
    refundUsage: () => quota.refund(userId),
  });
}

If the resume was already parsed, this returns the cached validated result without making another API call or reserving additional quota.

If schema validation fails, cachedLLMCall throws LLMError('validation') and does not write the response to the cache. This prevents invalid responses from becoming cached results.

reserveUsage and refundUsage are optional. Only pass them when you are tracking usage or enforcing quotas per user. Omit both if you do not need quota tracking.