Java SDK

A thin client for the governed chokepoint. If you can call it with java.net.http.HttpClient, you can call it with this: the SDK exists to save you the boilerplate of retries, typed errors and streaming, not to hide the API from you.

What this actually is, in plain terms

Forgebench sits in front of real model providers (OpenAI, Anthropic, Gemini, ...). Instead of your code holding an OpenAI key and calling OpenAI directly, it holds a Forgebench key (sk_...) and calls Forgebench, which forwards the request to whichever model you asked for. The client object below is your one doorway into that: every method on it is a different thing you can ask Forgebench to do on your behalf.

Doing it this way, instead of just calling OpenAI yourself, is what buys you the governed chokepoint: budget checks before money is spent, a tamper-evident audit row for every call, and a provider credential your code never has to hold. See Core concepts for the vocabulary (Agent, Budget, Policy, etc.) this whole SDK is built around.

Install

The SDK is built from the public forgebench-sdk repository by JitPack: add its repository and the dependency.

Maven:

<repositories>
  <repository><id>jitpack.io</id><url>https://jitpack.io</url></repository>
</repositories>

<dependency>
  <groupId>com.github.seedlinglabs</groupId>
  <artifactId>forgebench-sdk</artifactId>
  <version>main-SNAPSHOT</version>
</dependency>

Gradle:

repositories { maven { url 'https://jitpack.io' } }
dependencies { implementation 'com.github.seedlinglabs:forgebench-sdk:main-SNAPSHOT' }

main-SNAPSHOT always resolves to the latest stable release, the same as the no-ref pip/npm installs of the other SDKs. For a reproducible build, pin a release tag instead, for example java-v1.0.0; see Releases.

Requires Java 11+. The only runtime dependency is Jackson; HTTP is the JDK's own java.net.http.HttpClient. The packages are ai.forgebench (client and requests), ai.forgebench.types (responses) and ai.forgebench.errors.

Connect to the gateway

Every call, chat, agents, runs, tools, rides through one control-plane chokepoint: auth → tenant resolution (Postgres RLS) → budget pre-gate → per-tenant provider key injection → the gateway call → one metering event + one hash-chained audit row. You never send a provider credential (api_key/api_base for OpenAI, Anthropic, etc.); the SDK never has one to send, and the chokepoint strips a client-supplied one anyway.

import ai.forgebench.Forgebench;

Forgebench client = Forgebench.builder()
    .apiKey("sk_...")
    .baseUrl("https://api.forgebench.ai")
    .build();

With both environment variables set, Forgebench.builder().build() is all you need. For a self-hosted or local control plane, set FORGEBENCH_BASE_URL (or call .baseUrl(...)), for example http://localhost:8000 against a local stack. A wrong URL fails loudly (connection refused / 404), never silently.

The client is thread-safe: build one and share it, for example as a single Spring @Bean. Every error the SDK raises is unchecked, so calls compose inside lambdas without wrapping.

An asynchronous twin is available from the same builder: the same resources, each call returning a CompletableFuture. It runs calls on a daemon thread pool it owns (shut down by close()), or on yours with buildAsync(executorService):

import ai.forgebench.AsyncForgebench;
import ai.forgebench.ChatCompletionRequest;
import ai.forgebench.ChatMessage;
import ai.forgebench.Forgebench;

try (AsyncForgebench client = Forgebench.builder()
        .apiKey("sk_...")
        .baseUrl("https://api.forgebench.ai")
        .buildAsync()) {
    client.chat().completions().create(ChatCompletionRequest.builder()
            .model("mock-gpt")
            .addMessage(ChatMessage.user("hello"))
            .build())
        .thenAccept(resp -> System.out.println(resp.getContent()))
        .join();
}

A failed async call completes its future exceptionally with the same typed exception the blocking client throws (wrapped in a CompletionException by join()).

whoami

Confirms the key is valid and tells you what it resolves to: the same Principal every governed call is authorized against. Call this once at startup to fail fast on a bad credential, rather than discovering it on your first real request:

WhoAmI me = client.whoami();
System.out.println(me.getTenantId() + " " + me.getRoles() + " " + me.getScopes());

Chat completions

import ai.forgebench.ChatCompletionRequest;
import ai.forgebench.ChatMessage;
import ai.forgebench.types.ChatCompletion;

ChatCompletion resp = client.chat().completions().create(ChatCompletionRequest.builder()
    .model("mock-gpt")
    .addMessage(ChatMessage.user("Prove the governed path works."))
    .build());

System.out.println(resp.getChoices().get(0).getMessage().getContent());
System.out.println(resp.getUsage().getTotalTokens());
System.out.println(resp.getTraceId());
System.out.println(resp.getCallId());

resp.getContent() is a shortcut for the first choice's content. getTraceId() ties this one response to its audit entry, its metering record and its trace. getCallId() is this call's own row on the ledger; pass it as parentCallId on the calls it causes (see Call lineage below). Response fields this SDK version does not model are kept in resp.getExtra(). See Make your first call for the full response shape.

Messages are built with ChatMessage.system(...), user(...), assistant(...) and tool(...); a plain Map<String, Object> in the wire shape (role, content, tool_calls, tool_call_id, name) is accepted anywhere a message is. temperature(...) and maxTokens(...) have builder methods; toBuilder() copies a request to change one field for the next turn.

Streaming

import ai.forgebench.ChatCompletionStream;
import ai.forgebench.types.ChatCompletionChunk;

try (ChatCompletionStream stream = client.chat().completions().createStream(ChatCompletionRequest.builder()
        .model("mock-gpt")
        .addMessage(ChatMessage.user("Stream me a sentence."))
        .build())) {
    for (ChatCompletionChunk chunk : stream) {
        String delta = chunk.getChoices().get(0).getDelta().getContent();
        if (delta != null) System.out.print(delta);
    }
}

The budget gate still runs before the stream opens, so a 402 is thrown from createStream, never mid-stream. A 429 or a failure to connect before the stream opens is retried like any other write; a 5xx is not, and once events are flowing nothing is retried. Close the stream (try-with-resources) to release the connection if you stop early. On the streaming path the call's ledger id arrives in the X-Call-Id response header: read it with stream.getCallId().

Call lineage

Every governed response carries a call id. Pass it as parentCallId on the calls it causes, a sub-agent's turn, the next step of a tool loop, and the ledger records them as children: cost rolls up to the root, and the console draws the run as a tree instead of a flat set of rows sharing a trace. traceId groups; parentCallId structures. It's correlation only; it never affects whether a call is allowed or what it costs.

ChatCompletion plan = client.chat().completions().create(request);

ChatCompletion step = client.chat().completions().create(request.toBuilder()
    .parentCallId(plan.getCallId())
    .build());

To group several calls under one trace of your own, generate an id once per logical turn with Trace.newTraceId() and pass it as .traceId(...) on each of them.

Tools

There are two entirely separate gates for "may this agent call this tool," and which one applies depends on the tool's name in the model's tool_calls response. Sending the right shape matters more than anything else here: get the naming wrong and the gate that should authorize the call never sees it as one of its own.

Sending a tool schema

Pass the OpenAI-shaped tool list with .tools(...); tool_choice and every other OpenAI body field go through .extraBody(...), which the control plane forwards untouched to the provider. Below, search is a Tool Registry entry (bare name) and mcp_time_get_current_time is an MCP tool (mcp_ prefix):

ChatCompletion resp = client.chat().completions().create(ChatCompletionRequest.builder()
    .model("gpt-4o")
    .messages(messages)
    .tools(List.of(
        Map.of("type", "function", "function", Map.of(
            "name", "search",
            "description", "Search the KB",
            "parameters", Map.of("type", "object", "properties", Map.of("q", Map.of("type", "string"))))),
        Map.of("type", "function", "function", Map.of(
            "name", "mcp_time_get_current_time",
            "description", "Current time in a timezone",
            "parameters", Map.of("type", "object", "properties", Map.of("tz", Map.of("type", "string")))))))
    .extraBody(Map.of("tool_choice", "auto"))
    .build());

List<Map<String, Object>> calls = resp.getChoices().get(0).getMessage().getToolCalls();
for (Map<String, Object> call : calls == null ? List.<Map<String, Object>>of() : calls) {
    Map<?, ?> function = (Map<?, ?>) call.get("function");
    String name = (String) function.get("name");
    String arguments = (String) function.get("arguments");
    Object result = name.startsWith("mcp_")
        ? yourMcpClient.call(name, arguments)
        : yourToolDispatch(name, arguments);
}

getToolCalls() returns the OpenAI shape as plain maps ({"id", "type", "function": {"name", "arguments"}}), or null when the model called no tool; arguments is a JSON string, same as the raw API, not pre-parsed. Echo the turn back with ChatMessage.assistant(null, calls) and answer each call with ChatMessage.tool(callId, result) before the next turn.

A response naming a tool this agent isn't bound to never reaches you as-is: the whole response is refused with 403 tool_not_authorized (Tool Registry) or the equivalent MCP refusal; see Errors below. Bind the tool on the Tool Registry or MCP page first; no amount of retrying the identical call fixes an unbound tool.

Other fields only reachable through extraBody

tools has a builder method; the control plane's request model accepts several more fields that only reach it through this same extraBody passthrough (fields set by name always win over extraBody):

The agentTools() helper (agent credentials only)

If the API key this client was built with was issued to a registered agent (not a human/developer key), client.agentTools() gives you the agent's live MCP allowlist and a ready-made dispatch loop: the API key resolves the tool list server-side, so a binding revoked centrally drops out of the very next turn without you hardcoding or re-deploying anything. openaiSchema() builds the model's tool list from the live allowlist (GET /v1/agent-tools); dispatch then runs every tool_call the gate let through, reports each outcome, and returns the role: "tool" messages to append before the next turn:

List<Map<String, Object>> tools = client.agentTools().openaiSchema();

ChatCompletion turn = client.chat().completions().create(ChatCompletionRequest.builder()
    .model("gpt-4o")
    .messages(messages)
    .tools(tools)
    .build());

List<Map<String, Object>> toolMessages = client.agentTools().dispatch(
    turn.getChoices().get(0).getMessage().getToolCalls(),
    (name, args) -> yourMcpClient.call(name, args),
    turn.getCallId());

An exception from your executor is reported as that tool's error and surfaced to the model as {"error": ...}, so the model gets to decide what to do about a failed call, same as any other tool result. Pass typed argument schemas per tool with openaiSchema(Map.of("search", jsonSchema)); the control plane only stores a tool's name and description, not its input shape.

agentTools().report(callId, toolName, result, error, latencyMs) is the lower-level primitive dispatch calls internally; reach for it directly only if you're executing tools outside a simple executor callback.

Agents

Agents execute through the runs subsystem, which routes back through the same governed chokepoint: budget re-gated per hop, metered, audited:

import ai.forgebench.AgentCreateParams;
import ai.forgebench.RunParams;
import ai.forgebench.types.Agent;
import ai.forgebench.types.Run;

for (Agent a : client.agents().list()) {
    System.out.println(a.getId() + " " + a.getName() + " " + a.getModel());
}

Agent agent = client.agents().create(AgentCreateParams.builder()
    .name("Support bot")
    .ownerIdentityId(client.whoami().getIdentityId()) // the accountable owner: required
    .model("mock-gpt")
    .systemPrompt("You are a helpful support agent.")
    .build());

Run run = client.agents().run(agent.getId(), RunParams.builder()
    .input(Map.of("messages", List.of(
        Map.of("role", "user", "content", "Help me reset my password."))))
    .waitForCompletion(true)
    .timeout(Duration.ofSeconds(30))
    .build());
System.out.println(run.getStatus() + " " + run.getOutput());

ownerIdentityId is required: the control plane rejects an agent with no named owner, and the builder refuses to build without one.

Agent → agent (A2A tasks)

An agent bound to another by an operator can open a task on it through the door; the callee runs wherever it runs, and its own governed calls hang under the task in the ledger:

import ai.forgebench.CallParams;
import ai.forgebench.types.Task;

Task task = client.agents().call("qp-research", CallParams.text("10 MCQs on photosynthesis")
    .data(Map.of("grade", 9))
    .parentCallId(plan.getCallId())
    .build());
if (task.getState().equals("input_required")) {
    task = client.agents().call("qp-research", CallParams.text("CBSE")
        .contextId(task.getContextId())
        .build());
}
String notes = task.artifactText();

call blocks up to 30 seconds (.wait(Duration)) for an answer: completed, failed, or a question (input_required, read it with task.getQuestion()). A task still working after that is returned as-is; poll it with client.agents().task("qp-research", task.getId(), Duration.ofSeconds(20)) or stop it with cancel(...).

To BE a callee with no inbound port (pull delivery: this agent's own credential long-polls the door). Pass parentCallId(task.getCallId()) on any governed call made inside the handler, so it nests under the task, and client.agents().serve(handler) blocks, long-polling the door for this agent's tasks:

client.agents().serve(task -> {
    ChatCompletion answer = client.chat().completions().create(ChatCompletionRequest.builder()
        .model("mock-gpt")
        .addMessage(ChatMessage.user(task.getText()))
        .parentCallId(task.getCallId())
        .build());
    return task.done(answer.getContent());
});

The handler may return a TaskReply (task.done(...), task.ask(...), task.fail(...)), or just a String / Map / List of artifacts, which is coerced into a completed reply automatically. An exception fails that task with its message rather than ending the loop. ServeOptions sets the long-poll window, a task limit and a stop condition; interrupting the serving thread also ends the loop. In a server app, run serve on its own background thread.

Runs

Run run = client.runs().create(Map.of("prompt", "hello"));   // mock-gpt
run = client.runs().get(run.getId());
run = client.runs().waitFor(run.getId(), Duration.ofSeconds(30), Duration.ofMillis(500));
System.out.println(run.getStatus());

waitFor polls until the run reaches succeeded or failed, and throws a ForgebenchException if it is still running after the timeout. runs().create(agentId, input, model) runs an agent directly.

Tracing

Mark the steps of your agent and each run becomes one nested trace: the agent, its functions and tools, and every model call under the step that made it, with tokens and cost. Steps are sent in the background with the same API key; there are no other keys and nothing to configure. The SDK only writes traces; read them in the console. The full guide is Tracing your agent.

Steps with try-with-resources

startActiveObservation opens a step that is current until it is closed. Steps and model calls made on the same thread inside it nest under it, and it ends even when an exception is thrown.

import ai.forgebench.tracing.Observation;
import ai.forgebench.tracing.ObservationType;
import ai.forgebench.tracing.TraceAttributes;
import ai.forgebench.tracing.Tracing;

Tracing tracing = client.tracing();

try (Observation agent = tracing.startActiveObservation(ObservationType.AGENT, "support-agent")) {
    tracing.updateCurrentTrace(TraceAttributes.builder()
            .userId("customer-42").sessionId("chat-7").tags("support").build());
    agent.input(question);
    ChatCompletion reply = client.chat().completions().create(request);   // nested
    agent.output(reply.getContent());
}

ObservationType is SPAN, AGENT, TOOL, CHAIN, RETRIEVER, EMBEDDING, EVALUATOR, GUARDRAIL or EVENT; names are free text.

One call as a step

tracing.observe runs a call as a step: the return value is its output, and an exception is recorded as an error and re-thrown unchanged (checked exceptions too). tracing.run is the same for a call with no result.

Order order = tracing.observe(ObservationType.TOOL, "lookup_order", "A-1001",
        () -> orders.get("A-1001"));

tracing.run(ObservationType.TOOL, "send_receipt", () -> mailer.send(order));

Values are captured when they are set, serialised with Jackson (getters are called), and cut at 64,000 characters.

Steps that start and end in different places

tracing.startObservation(...) creates a step that does not become current, for a callback or a background job; step.startObservation(...) adds a child. Call end() when the work is done; a step that never ends is never sent.

Observation job = tracing.startObservation(ObservationType.CHAIN, "nightly-sync");
Observation page = job.startObservation(ObservationType.RETRIEVER, "fetch-page");
page.output(Map.of("rows", 120));
page.end();
job.end();

Threads and executors

The current step belongs to the thread. Hand work to an executor with Tracing.wrap(...); AsyncForgebench does this for every call it makes.

Future<Order> order = executor.submit(Tracing.wrap(() -> loadOrder("A-1001")));
CompletableFuture<Order> later = CompletableFuture.supplyAsync(
        Tracing.wrapSupplier(() -> loadOrder("A-1002")), executor);

Spring AI

Spring AI runs a model's tool loop on the calling thread, so its model calls nest by themselves. Wrap each ToolCallback so every tool run is a step, and open one step per request:

class TracedTool implements ToolCallback {
    private final ToolCallback delegate;
    private final Tracing tracing;

    TracedTool(ToolCallback delegate, Tracing tracing) {
        this.delegate = delegate;
        this.tracing = tracing;
    }

    @Override public ToolDefinition getToolDefinition() { return delegate.getToolDefinition(); }
    @Override public ToolMetadata getToolMetadata() { return delegate.getToolMetadata(); }
    @Override public String call(String input) { return call(input, null); }

    @Override
    public String call(String input, ToolContext context) {
        return tracing.observe(ObservationType.TOOL, delegate.getToolDefinition().name(), input,
                () -> delegate.call(input, context));
    }
}

Configuration

Forgebench client = Forgebench.builder()
        .tracing(TracingConfig.builder()
                .sampleRate(0.2)            // keep 20% of traces (whole traces, never half)
                .environment("production")
                .release("2.3.0")
                .mask(myMask)               // UnaryOperator<JsonNode>, run on every value and error message
                .build())
        .build();

Without .tracing(...), the client reads FORGEBENCH_TRACING_ENABLED, FORGEBENCH_SAMPLE_RATE, FORGEBENCH_ENVIRONMENT and FORGEBENCH_RELEASE; a value that does not parse is ignored with a warning. enabled(false) turns tracing into a no-op without touching your code.

Sending before exit

Steps go out every second or so from one daemon thread per client. A short program should call client.flush() before it ends; client.close() sends what is left and stops, and a JVM shutdown hook flushes as a safety net. A failed send is logged (System.Logger "forgebench.tracing") and dropped, never thrown.

Errors

Every SDK-raised error is an unchecked ai.forgebench.errors.ForgebenchException. Network failures throw ForgebenchConnectionException directly; anything that got an HTTP response throws a subclass of ApiException, one per status code, each carrying getStatusCode(), getCode(), getDetail(), getBody() and getRequestId():

A RateLimitException has already been retried with backoff up to maxRetries (default 2) before the SDK throws it; by the time you see one, the SDK has given up. The same goes for a ServerException or a connection failure on a read. A write (a chat call, an A2A task, a tool report) is never replayed after a 5xx or a dropped connection: the server may already have run and billed it, and it has no idempotency key, so a ServerException from a chat call means "may or may not have run": check before repeating it. Every other error means "this exact call will fail again unchanged"; retrying it in a loop just produces the identical refusal, and for a tool call specifically, three repeated unauthorized attempts inside ten minutes escalate to their own 429 (tool_repeatedly_unauthorized) precisely to stop that pattern before it burns another provider call. getCode() carries the cause when the server names one (tool_not_authorized, not_permitted, agent_budget_exceeded, ...); when it comes back null, the full parsed response is in getBody(). A 402's body carries the numbers behind the refusal (limit_usd, spent_usd, attempted_cost_usd):

import ai.forgebench.errors.*;

try {
    ChatCompletion resp = client.chat().completions().create(request);
} catch (BudgetExceededException e) {
    log.warn("budget gate refused the call: {}", e.getBody());
} catch (PermissionDeniedException e) {
    log.error("not authorized ({}): {}", e.getCode(), e.getDetail());
} catch (AuthenticationException e) {
    log.error("auth failed: {}", e.getDetail());
} catch (ValidationException e) {
    log.error("request rejected: {}", e.getBody());
} catch (NotFoundException | ConflictException e) {
    log.error("{}", e.toString());
} catch (RateLimitException e) {
    log.warn("rate limited after retries: {}", e.getCode());
} catch (ServerException e) {
    log.error("control-plane error after retries (request {})", e.getRequestId());
} catch (ForgebenchConnectionException e) {
    log.error("could not reach the control plane: {}", e.getMessage());
}

Metering and budget state

MeteringSummary s = client.meteringSummary();
System.out.printf("spent $%s / limit $%s (remaining $%s)%n",
    s.getSpentUsd(), s.getMonthlyLimitUsd(), s.getRemainingUsd());
System.out.println(s.getByModel());

The workspace's spend is billing data: it needs a key with the admin role or the billing:read permission. An agent key gets a PermissionDeniedException.

Next