Skip to content

LLM APIs ​

The wvlet.uni.ai packages of uni give Scala code typed requests, responses, and streaming events for the LLM APIs that apps call most:

PackageAPIHosts
wvlet.uni.ai.anthropicMessages (/v1/messages)Anthropic API, Claude on Vertex AI, Claude on Bedrock via the Anthropic API
wvlet.uni.ai.openaiResponses (/v1/responses) and Chat CompletionsOpenAI, OpenAI models on Bedrock, and OpenAI-compatible servers such as vLLM and Ollama
wvlet.uni.ai.geminigenerateContent and streamGenerateContentGoogle AI Studio, Vertex AI
wvlet.uni.aiA vendor-neutral model and converters

The types use Scala names, and their codecs write each vendor's exact JSON: max_tokens, type: "tool_use", Gemini's {"functionCall": ...} parts, no null for absent fields. Block, item, and event types a release doesn't know yet decode to an Unknown case that keeps the raw JSON, and unmodeled request fields go in extra, so a newer payload passes through unchanged. It runs on the JVM, Scala.js, and Scala Native, and comes with the uni artifact (Installation); code you don't call is removed when Scala.js and Scala Native link.

Calling a vendor API ​

Each vendor client returns an Rx of the response, or an Rx of stream events:

scala
import wvlet.uni.ai.anthropic.*
import wvlet.uni.rx.Rx

val claude = AnthropicClient(AnthropicConfig.fromEnv) // ANTHROPIC_API_KEY

val request = MessagesRequest(
  model = "claude-sonnet-4-5",
  maxTokens = 1024,
  messages = Seq(Message.user("Name three Scala build tools"))
).withSystem("Answer in one line.")

val reply: Rx[MessagesResponse] = claude.messages(request)
reply.map(r => println(r.text))

// Streaming: one StreamEvent per SSE event
val text: Rx[String] = claude.stream(request).flatMap(e => Rx.fromSeq(e.textDelta.toSeq))

MessageAccumulator folds the events into the final MessagesResponse, joining text, thinking, and tool-input deltas. The other vendors have the same pair of calls:

scala
import wvlet.uni.ai.openai.*

val openai = OpenAIClient(OpenAIConfig.fromEnv) // OPENAI_API_KEY, OPENAI_BASE_URL

val res = openai.responses(ResponsesRequest("gpt-5.1", Seq(Item.user("Hello"))))
val events: Rx[ResponseStreamEvent] = openai.streamResponses(
  ResponsesRequest("gpt-5.1", Seq(Item.user("Hello")))
)

// Any OpenAI-compatible server
val local = OpenAIClient(OpenAIConfig(apiKey = None, baseUrl = "http://localhost:8000/v1"))
val chunks: Rx[ChatCompletionChunk] = local.streamChatCompletions(
  ChatCompletionRequest("Qwen/Qwen3-8B", Seq(ChatMessage.user("Hello")))
)
scala
import wvlet.uni.ai.gemini.*

val gemini = GeminiClient(GeminiConfig.fromEnv) // GEMINI_API_KEY
val g      = gemini.generateContent(
  "gemini-2.5-flash",
  GenerateContentRequest(Seq(Content.user("Hello")))
)

// Vertex AI with Application Default Credentials
val vertex = GeminiClient(GeminiConfig(GeminiHost.Vertex("my-project", "global")))

Amazon Bedrock serves Claude through the Anthropic Messages API and its OpenAI models through the OpenAI API, both on bedrock-runtime.<region>.amazonaws.com. Use AnthropicClient or OpenAIClient with a Bedrock host; requests, streams, and converters are the same as on the vendors' own APIs:

scala
import wvlet.uni.ai.BedrockAuth
import wvlet.uni.ai.anthropic.*
import wvlet.uni.ai.openai.*

// AWS_BEARER_TOKEN_BEDROCK if set, else the default AWS credentials chain (SigV4)
val claude = AnthropicClient(AnthropicConfig(AnthropicHost.Bedrock("us-east-1")))
claude.stream(MessagesRequest("us.anthropic.claude-sonnet-5", 1024, Seq(Message.user("Hello"))))

val gpt = OpenAIClient(OpenAIConfig.bedrock("us-east-1"))
gpt.streamResponses(ResponsesRequest("us.openai.gpt-5.6-sol", Seq(Item.user("Hello"))))

Claude on Vertex AI also takes the Anthropic request body: use AnthropicHost.Vertex(projectId, region), and the client moves the model into the URL and adds Vertex's anthropic_version. countTokens works on the Anthropic API only.

A failed call ends the Rx with an LlmApiException carrying the HTTP status, the vendor's error type (overloaded_error, rate_limit_error, RESOURCE_EXHAUSTED, ThrottlingException), and the body. The clients don't retry: a retried completion is billed twice, so retry where you know that is safe, with Retry.

Converting between vendors ​

wvlet.uni.ai has a vendor-neutral model: LlmRequest, LlmResponse, and a stream of LlmStreamParts that open, grow, and close text, reasoning, and tool-input blocks. Each vendor package has a converter to and from it, so any vendor's request can be served by any other vendor. Every client also implements LlmClient, which takes an LlmRequest:

scala
import wvlet.uni.ai.*
import wvlet.uni.ai.anthropic.*
import wvlet.uni.ai.openai.{OpenAIClient, OpenAIConfig}
import wvlet.uni.rx.Rx

// A Claude Code request, served by an OpenAI model
def serveWithOpenAI(body: String, openai: OpenAIClient): Rx[StreamEvent] =
  val request = JsonCodec[MessagesRequest].fromJson(body)
  val neutral = AnthropicConverter.toLlmRequest(request).withModel("gpt-5.1")
  val encoder = AnthropicConverter.StreamEncoder()
  openai
    .stream(neutral)
    .flatMap(part => Rx.fromSeq(encoder.encode(part)))
    .concat(Rx.fromSeq(encoder.finish()))

The encoder adds the framing Anthropic clients expect: message_start first, a start and stop around every block, and message_delta with the stop reason and usage at the end. Send the events with Response.eventStream(events.map(_.toServerSentEvent)) from a REST server. The other directions work the same way, with ResponsesConverter, ChatCompletionsConverter, and GeminiConverter, each with a StreamDecoder and a StreamEncoder.

What the converters take care of:

  • Usage: Anthropic counts input tokens without cache reads and writes; OpenAI and Gemini include them. Gemini counts thoughts apart from output. LlmUsage always includes both, and each converter adjusts.
  • Stop reasons: Gemini says STOP and Responses says completed even when the model called a tool, so the converters derive tool_use / tool_calls from the content. Chat Completions sends usage after the finish chunk, so its decoder waits for both before the final part.
  • Reasoning across vendors: Claude Code returns thinking blocks unchanged, signatures included. The Anthropic encoder puts OpenAI encrypted reasoning and Gemini thought signatures in the thinking signature, with a wvlet.uni.ai: prefix, and the decoder restores them on the next turn. The OpenAI encoder carries Anthropic signatures in encrypted_content the same way. Gemini function calls in history without a signature get Gemini's documented placeholder.
  • Tool ids and schemas: ids are rewritten deterministically where a vendor restricts them (Anthropic). Gemini gets the JSON Schema unchanged through parametersJsonSchema.
  • Vendor-only blocks: server tool results, built-in tools, and unknown blocks travel as LlmPart.Raw and LlmTool.Provider, and only reach the vendor they came from.

Options only one vendor understands go in providerOptions, keyed by vendor name; the converter for that vendor merges them into the request body:

scala
import wvlet.uni.json.JSON.JSONObject

val withOptions = LlmRequest("gpt-5.1", Seq(LlmMessage.user("Hi"))).copy(
  providerOptions = ProviderMetadata.of(ProviderMetadata.OpenAI, Json.field("service_tier", "flex"))
)

Serving an API ​

The types also describe what a server sends. To stream Anthropic events from a handler, map them to SSE events:

scala
import wvlet.uni.http.{Http, Request, Response}

def messagesHandler(openai: OpenAIClient)(req: Request): Rx[Response] =
  Rx.single(Response.eventStream(serveWithOpenAI(req.content.toContentString, openai).map(_.toServerSentEvent)))

Chat Completions streams end with ChatCompletionChunk.done (data: [DONE]).

Every model type has a JsonCodec with toJson and fromJson, and codec.weaver gives a Weaver for returning the types from routers and RPC handlers.

Released under the Apache 2.0 License.