Uni 2026.3.0: Lakehouse Tables in Pure Scala, Typed LLM APIs, and One Rx Event Model
Uni 2026.3.0 turns its Parquet reader into a lakehouse client. It reads folders and bucket prefixes of Parquet files as one table, reads Delta Lake and Apache Iceberg tables, and writes Iceberg tables with atomic commits. On a 1M-row benchmark, its Iceberg writer keeps pace with or beats DuckDB's. It is all pure Scala, so it runs on the JVM, in the browser and Node.js with Scala.js, and in native binaries with Scala Native.
The release also adds typed APIs for Anthropic, OpenAI, and Gemini with OpenTelemetry tracing, and gives Rx streams one event model. It moves a few packages, so check Upgrading before you bump the version.
resolvers += "wvlet" at "https://repo.wvlet.org/maven2"
libraryDependencies += "org.wvlet.uni" %% "uni" % "2026.3.2"Uni releases go to the wvlet Maven repository first. 2026.3.0 is not on Maven Central yet. On npm, the TypeScript package is @wvlet/uni.
Parquet datasets, Delta Lake, and Iceberg
Data that Spark, Trino, DuckDB, or Athena writes is rarely one file. It is many files under a folder or a bucket prefix, split into key=value folders, and often tracked by a Delta Lake or Iceberg log. Parquet.dataset reads all of these as one table, from a local directory or an s3://, gs://, or az:// prefix:
import wvlet.uni.parquet.*
import wvlet.uni.parquet.ParquetFilter.*
import wvlet.uni.weaver.Weaver
import java.time.LocalDate
case class Visit(id: Long, user: String, dt: LocalDate, country: String) derives Weaver
val ds = Parquet.dataset("s3://my-bucket/visits/")
try
val recentJP = ParquetReadOptions
.default
.withFilter(col("dt") >= "2026-11-15" && col("country") === "JP")
// Skips the files and row groups the filter rules out, then filters rows
val visits = ds.readAs[Visit](recentJP).toSeq
finally ds.close()- Hive-style partitions.
dt=2026-11-20/country=JP/folders become columns. Files the filter rules out are never opened. Globs (s3://bucket/t/dt=2026-11-*/) and explicit file lists work too. - Files that drifted. Columns are matched by name across files, so a column missing from an older file reads as null. INT32 widens to INT64 or DOUBLE, and FLOAT to DOUBLE, including inside structs, lists, and maps.
- Exact row filters.
ParquetFilterskips row groups by their statistics and then filters rows exactly, with SQL's rules for nulls. - Delta Lake and Iceberg. A root with
_delta_log/reads as a Delta table, and a root withmetadata/*.metadata.jsonreads as an Iceberg table. Either way, only the files of the current version are read, so deleted rows stay deleted.withVersion(n)andwithSnapshotId(id)read older versions.
Writing datasets and Iceberg tables
Parquet.writeDataset writes the other way. It splits rows into key=value folders, can sort each partition (withSortBy), and publishes each file only once it's finished. With ParquetTableFormat.Iceberg, it writes an Iceberg table that Spark, Trino, DuckDB, and pyiceberg read:
val asIceberg = ParquetDatasetWriteOptions
.default
.withTableFormat(ParquetTableFormat.Iceberg)
.withPartitionBy("dt")
val visitRows = Seq(Visit(1, "alice", LocalDate.parse("2026-11-21"), "JP"))
Parquet.writeDataset("s3://my-bucket/warehouse/visits/", visitRows, asIceberg) // creates the table
Parquet.writeDataset(
"s3://my-bucket/warehouse/visits/",
visitRows,
asIceberg.withMode(ParquetWriteMode.Append) // commits a new snapshot
)Each commit creates the next metadata/vN.metadata.json only if it doesn't already exist. That uses S3's If-None-Match: *, GCS's ifGenerationMatch=0, or Azure's If-None-Match: *, so two writers can't both commit the same version. The losing writer reloads the table and retries. Manifests carry each file's row counts, null counts, and lower and upper bounds, so engines can skip files.
Uni commits only to Hadoop-style tables, which keep their metadata in metadata/ and need no catalog. Tables owned by a Glue, REST, or Hive catalog are refused, so a write can't fork them. Overwrite modes, partition transforms other than identity, and schema evolution are still to come.
The same dataset reader and writer are in TypeScript. openDataset and writeDataset in @wvlet/uni/parquet work in Node.js and in browsers, with async calls only.
Benchmarked and checked with DuckDB
DuckDB is the tool we reach for to query Parquet and Iceberg, so we used it as the reference. We benchmarked Uni's Iceberg writer against DuckDB 2.0.1, which writes Iceberg through a REST catalog. Both sides wrote the same 1M rows of TPC-H-style lineitem data, already in memory, as column batches:
| Iceberg write, 1M rows | Uni on the JVM, 8 threads | DuckDB, 18 threads | Uni, 1 thread | DuckDB, 1 thread |
|---|---|---|---|---|
| Unpartitioned, Snappy | 120 ms | 217 ms | 252 ms | 269 ms |
| Partitioned by ship mode, Snappy | 97 ms | 136 ms | 344 ms | 391 ms |
| Unpartitioned, ZSTD | 256 ms | 270 ms | 452 ms | 313 ms |
| Partitioned by ship mode, ZSTD | 122 ms | 143 ms | 549 ms | 453 ms |
Three changes got it there:
- Partition values without per-row work. Earlier, the writer rebuilt the partition folder name on almost every row, and it decoded every field name to find the partition columns.
- Dictionaries only where they pay off. Like parquet-java, the writer now drops a column's dictionary if it isn't smaller than plain encoding after 20,000 values. This made the partitioned tables 13-18% smaller.
- Several threads on the JVM. A batch's columns are encoded and compressed in parallel, and so are the partitions of a batch.
ParquetWriteOptions.withParallelism(n)sets the thread count, and the bytes written are the same at every setting.
ZSTD is the one case where single-threaded DuckDB still wins. Uni's ZSTD compressor is its own pure-Scala code, which is slower than libzstd. Writes from case classes still turn rows into columns on the calling thread.
Speed only counts if the tables are right, so we read every benchmark table back with DuckDB's iceberg_scan and with pyiceberg:
- Both readers return exactly the source rows, duplicates included.
- Every manifest entry's row count, size, partition value, null counts, and bounds match its data file.
- pyiceberg reads a single file for a one-partition filter.
The benchmark and the check are in examples/parquet-bench, so you can run them yourself.
Typed LLM APIs
The wvlet.uni.ai packages now have typed requests, responses, and streaming events for the Anthropic Messages API, the OpenAI Responses and Chat Completions APIs, and Gemini:
import wvlet.uni.ai.anthropic.*
import wvlet.uni.rx.Rx
val claude = AnthropicClient(AnthropicConfig.fromEnv) // ANTHROPIC_API_KEY
val request = MessagesRequest(
model = "claude-sonnet-4-5",
maxTokens = 1024,
messages = Seq(Message.user("Name three Scala build tools"))
)
val text: Rx[String] = claude.stream(request).flatMap(e => Rx.fromSeq(e.textDelta.toSeq))- Each vendor's exact JSON. Types use Scala names, and their codecs write each vendor's wire format. Fields and events a release doesn't model yet pass through unchanged.
- Converters between vendors. One vendor's request can be served by another vendor's API.
- Bedrock through the vendors' own APIs. Claude on Bedrock uses the Anthropic Messages API, and OpenAI models on Bedrock use the OpenAI API. Both sign with SigV4 or use a Bedrock API key.
- Tracing. An HTTP client with telemetry turns each LLM or MCP call into an OpenTelemetry GenAI span. The span records the model, token usage, and finish reasons, along with
gen_ai.client.inference.*duration and token metrics. Prompts and completions are never recorded.
The same release traces Uni's HTTP server, router, RPC dispatcher, and HTTP client on every platform. They read and write the W3C traceparent header, so a request stays in one trace across services, and the traces can go to Datadog or any backend over OTLP. See Telemetry.
One Rx event model
Rx streams now follow the event model of R3. OnNext carries a value, OnError reports an error the stream continues past, and a stream ends at most once, with OnCompletion or OnFailure(e):
import wvlet.uni.rx.*
RxRunner.runContinuously(Rx.fromSeq(Seq(1, 2, 3))) {
case OnNext(v) => println(s"value: ${v}")
case OnError(e) => println(s"error, the stream goes on: ${e.getMessage}")
case OnCompletion => println("completed")
case OnFailure(e) => println(s"failed: ${e.getMessage}")
}run, runOnce, and runContinuously now share one runner, so an operator behaves the same way whichever one you use. The rules are checked against models in Lean, and the test suite replays cases generated from those models. Stream Events lists the rules.
The uni-ui gallery
The docs now have a live UI gallery for uni-ui:
- Every component. Each one has live examples next to the Scala code that renders them, plus its parts, API, and an accessibility check.
- Code and preview linked. Hovering a line of code outlines what it creates, and Inspect finds the line behind any part of the preview.
- Rx pages. An Rx group draws each stream's events on a timeline as you click and type.
- New layout components.
Stack,Grid,GridItem, andSeparator.
Upgrading
Uni is still young, so packages move without deprecated aliases. These moves fail at compile time:
| Before | In 2026.3.0 |
|---|---|
The uni-parquet artifact | Part of uni (the wvlet.uni.parquet package is unchanged); drop the dependency |
wvlet.uni.mcp.* | wvlet.uni.ai.mcp.* |
wvlet.uni.mcp.description | wvlet.uni.surface.description |
wvlet.uni.agent.UiAgentApi | wvlet.uni.ui.agent.UiAgentApi |
AWS, Google, and Azure credentials, SigV4 (in wvlet.uni.storage) | wvlet.uni.cloud.* |
wvlet.uni.config.SettingStore and its backends | wvlet.uni.storage.* |
OnCompletion(Result.Success(_)), OnCompletion.success | OnCompletion |
OnCompletion(Result.Failure(e)), OnCompletion.failure(e) | OnFailure(e) |
c: OnCompletion type patterns | c: OnEnd |
The wvlet.uni.ai.bedrock Converse client | AnthropicClient or OpenAIClient with a Bedrock host |
Two changes don't fail to compile:
- Rx subscriber errors are logged.
subscribe,runContinuously, andRxVar.foreachnow log an error that nobody handled, as a warning. They used to throw it. - Credential errors are
CloudExceptions. A credential failure is now aCloudException, the new parent ofStorageException. CatchCloudExceptionto handle both kinds of failure.
The full list of changes is in the release notes.