Eight modules. One install.

Fan out many prompts as parallel encrypted inferences with a capped concurrency. Stable result order, per-slot errors so one stalled worker does not kill the batch. Optional onSlotComplete for live progress UI, optional AbortSignal to cancel queued work.

import { runInferenceBatch } from "lightnode-sdk";

const results = await runInferenceBatch({
  network: "testnet",
  privateKey: process.env.PRIVATE_KEY!,
  model: "llama3-8b",
  system: "Reply in one short sentence.",
  concurrency: 4,
  prompts: [
    "fact about the ocean",
    "fact about the moon",
    "fact about coffee",
  ],
  onSlotComplete: ({ index, result, error }) => {
    console.log(`#${index}`, error?.message ?? result?.answer);
  },
});

Fits: batch evals, content scoring, RAG re-ranking, parallel rewrites.

ReAct-style loop: model thinks, picks a tool, runs it, observes the result, iterates. Uses simple string markers (<tool> / <answer>) so it works on small open models like llama3-8b without native function-calling support.

import { Agent } from "lightnode-sdk";

const agent = new Agent({
  network: "testnet",
  privateKey: process.env.PRIVATE_KEY!,
  model: "llama3-8b",
  system: "You are a careful research assistant.",
  tools: [{
    name: "add",
    description: "Add two integers, return the sum.",
    args: { a: "int", b: "int" },
    handler: ({ a, b }) => Number(a) + Number(b),
  }],
  maxIterations: 4,
});

const { answer, steps } = await agent.run("17 + 25?");

Fits: autonomous tasks, search + summarize, lookup chains, multi-step reasoning with deterministic side-effects.

Pass a network ID, a private key, a prompt. The SDK builds the viem clients, runs the SIWE handshake, opens the encrypted session, submits, decrypts. The example in the quickstart-inference folder is this.

You already wire up wagmi or a server-side viem WalletClient. Pass the gateway client (with bearer) plus the wallet and public clients. Same retry, streaming, and proof chain as the high tier.

Call prepareSession to pick a worker and get a wrapped session key. Sign createSession yourself. Encrypt + upload with submitPrompt. Sign submitJob. Decrypt frames with decryptResponse. Plus the typed errors below if you want to recover differently than the default retry policy.