Infernet Docs

The inference control plane for agent-scale workloads. Point your agents at one endpoint; Infernet routes, batches, caps and traces every request across every model. You keep your own provider keys — Infernet never resells model access.

Quickstart

Install the SDK, set your Infernet key, and route your first completion. Your existing model-provider keys stay in Infernet's vault or your own secret store — the call below never marks up a token.

# TypeScript / Node
npm install @infernet/sdk

# Python
pip install infernet

import { Infernet } from "@infernet/sdk";

const fabric = new Infernet({ apiKey: process.env.INFERNET_KEY });

const res = await fabric.completions.create({
  model:  "auto",            // let the policy pick
  policy: "lowest-cost-under-latency",
  ceiling:{ usd: 18.0 },     // hard budget cap
  messages:[{ role:"user", content:"Draft a resolution for this ticket." }],
});

console.log(res.choices[0].message.content);
console.log(res.usage.routedTo, res.usage.costUsd); // full accounting

Core concepts

• Fabric — the control plane sitting between your agents and the models.
• Route — a named request path with its own policy, ceiling and trace scope.
• Policy — declarative rules (cost, latency, quality floor, fallback order) evaluated per request.
• Ceiling — a hard token/dollar cap; when hit, the fabric degrades gracefully instead of overspending.

Routing policies

A policy tells the fabric how to choose a model for each request. Declare it in config, not code — swapping strategy never touches your agent logic.

route: agent-step
policy:
  strategy: lowest-cost-under-latency
  latency_budget_ms: 600
  quality_floor: 3
  fallback: [ opus-4, llama-4-70b ]
ceiling:
  tokens: 2000000
  usd: 18.00
  on_exceed: downgrade   # downgrade | throttle | reject

API reference

The HTTP API is OpenAI-compatible at the message layer, with Infernet routing fields added. Base URL:

POST https://api.infernet.app/v1/completions
Authorization: Bearer $INFERNET_KEY
Content-Type: application/json

{
  "model": "auto",
  "policy": "lowest-cost-under-latency",
  "ceiling": { "usd": 18.0 },
  "messages": [{ "role": "user", "content": "..." }]
}

SDK reference

First-class SDKs for TypeScript, Python and Go — one typed interface across every model.

from infernet import Infernet

fabric = Infernet(api_key=os.environ["INFERNET_KEY"])

res = fabric.completions.create(
    model="auto",
    policy="highest-quality",
    ceiling={"usd": 40},
    messages=[{"role": "user", "content": "Synthesize the attached runbook."}],
)
print(res.choices[0].message.content, res.usage.routed_to)

import "github.com/infernet-ai/infernet-go"

fabric := infernet.New(os.Getenv("INFERNET_KEY"))
res, _ := fabric.Completions.Create(ctx, infernet.Request{
    Model:  "auto",
    Policy: "lowest-latency",
    Messages: []infernet.Message{{Role:"user", Content: prompt}},
})

Cost ceilings

Ceilings are the safety net for autonomous agents. Set a hard cap per route, tenant or workspace. When a request would cross it, choose the degradation mode:

• downgrade — reroute to the cheapest eligible model and continue.
• throttle — queue and pace requests to stay under the rate.
• reject — return a typed CeilingExceeded error your agent can catch.

Traces & observability

Every token, hop and retry is captured as an OpenTelemetry span. Replay any agent run, attribute spend down to the line of code, and export to the stack you already run.

infernet traces export --route agent-step --last 24h \
  --otlp https://collector.internal:4317

Security & residency

• Bring your own keys — provider credentials stay in your vault; Infernet orchestrates, never resells.
• Edge PII redaction — sensitive fields are scrubbed before a request leaves your region.
• Region residency — pin all processing to a residency region on Enterprise plans.
• SOC 2 aligned — audit logs, RBAC and a DPA available for regulated teams.

About Infernet

Infernet AI is built and operated by INFERNET AI PTE. LTD., registered in Singapore (UEN 202506302K). We build the missing layer between agents and models — the control plane that makes agent-scale inference fast, affordable and governable — without ever standing between you and your own provider accounts.

Get in touch: hello@infernet.app