DOCSBeta
7Log in

Inference

A workflow calls a model with agent(). Boardwalk is model-agnostic: it routesinference, it doesn't serve it. You can let Boardwalk pick a model with no API key, name a specific model, or point a call at your own provider and key.

Model is per call

The model is chosen on each agent() call, not on the workflow. meta declares no model and no provider. A workflow that does no model work names none, and one that calls three different models is just the obvious code:

const draft  = await agent("Write release notes from this diff: ...");
const review = await agent("Critique these notes harshly: ...", {
  model: "anthropic/claude-sonnet-4.5",
});

Managed inference

Call agent(prompt) with no options and Boardwalk fulfills it on the managed lane, with no API key and no model menu. Omit the model and the Auto lane picks one for you per request:

const summary = await agent("Summarize this thread for a busy exec: ...");

Managed inference is the default provider (named boardwalk). Boardwalk meters the tokens and bills your org. See Pricing for how a run is priced.

Auto

Don't want to name a model? Pass auto (or omit modelon the managed lane) and Boardwalk routes each call to a strong default for that prompt. The response reports the model that actually served it, and you're billed at that model's normal rate, with no routing fee:

await agent(prompt, { model: "auto" });

auto is a moving target by design: as stronger models ship, the default it resolves to improves without a change to your program. Name a model explicitly when a call needs a specific one (reproducibility, a known strength, or a price ceiling). See the models page for every id you can pin.

Pick a model

Pass model to choose one. It's an opaque <vendor>/<model> string: the vendor prefix names the model, never your credentials. Under the default provider, a named model is the managed lane serving that model (still no key of yours):

await agent(prompt, { model: "anthropic/claude-sonnet-4.5" });
await agent(prompt, { model: "openai/gpt-4o" });

The managed catalog is a curated list, not the whole internet of models. See the models page for every supported id and its live rate. An unknown id fails that one agent() call with a clear error; the rest of the run continues.

Reasoning

Control how hard a model thinks before answering with reasoning on the agent() call. A bare string is an effort level, "none" through "xhigh"; the object form instead takes a direct maxTokens cap, and excludekeeps the reasoning trace out of the response. Omit it for the provider's adaptive default.

await agent("Prove this step by step: ...", { reasoning: "high" });
await agent(prompt, { reasoning: { maxTokens: 8000, exclude: true } });

It is one neutral control that maps to whatever knob the serving model exposes (reasoning_effortfor an OpenAI-compatible endpoint, a thinking-token budget for Anthropic and Bedrock), so the same call works whichever model serves it. A model that doesn't support a level returns an error rather than silently downgrading.

Prompt caching

Managed inference caches the stable front of your prompt automatically, with nothing to configure. The benefit shows up across the turns of a single agent()tool-use loop: the instructions, tool definitions, and early context stay the same, so after the first turn the model reads that prefix from cache at a small fraction of the normal input cost. Keep the fixed part of a prompt at the front and put what varies last, and don't bake anything that changes every run (a timestamp, a random id) into the instructions, or nothing is reused. A one-shot call with no follow-up turn doesn't cache, which is fine: make those cheap with a small model and a tight prompt. See Writing efficient workflows for the full guidance.

Bring your own provider

To use your own key (a vendor account or any OpenAI-compatible endpoint like vLLM, Together, a local Ollama, or Bedrock), name a provider other than boardwalk. Boardwalk never reaches for your key unless a call asks for it by provider name:

await agent(prompt, { model: "llama-3.3-70b", provider: "my-vllm" });

On Boardwalk, a provider is configured once in the dashboard (its base URL and which secret holds the key); the key lives in the secrets vault and is resolved per run, never exposed to the model loop. You pay that vendor directly. BYO keeps inference cost off Boardwalk's bill.

Across the three engines

The same agent() call resolves a little differently per engine, and nothing in your program changes:

  • Boardwalk (deployed):the managed lane just works, no key. Named providers resolve from the org's configured providers and secrets.
  • boardwalk dev: the default boardwalk provider reaches managed inference using your boardwalk login account. Logged out, name a provider with your own key instead, and the error tells you exactly what to set.
  • Self-hosting: bring your own by default (declare providers + keys), or set BOARDWALK_API_KEY to use managed inference. See Self-hosting.

Structured output

Pass a schema (JSON Schema) and agent() resolves to a validated object instead of text. Pass the expected type for inference at the call site:

type Bug = { title: string; severity: "low" | "high" };

const bugs = await agent<Bug[]>("Find the bugs in this diff: ...", {
  schema: {
    type: "array",
    items: {
      type: "object",
      properties: {
        title: { type: "string" },
        severity: { type: "string", enum: ["low", "high"] },
      },
      required: ["title", "severity"],
    },
  },
});

See Writing workflows for agent() in context and the SDK reference for the full AgentOptions.