Telemetry
Uni services can report traces to Datadog, or to any backend that speaks OpenTelemetry, in two ways:
- Attach a JVM agent. You change nothing in the code, but it works only on the JVM and only for libraries the agent knows.
- Turn on Uni's built-in tracing. Uni's own HTTP server, router, RPC dispatcher, and HTTP client then create spans on every platform: JVM, Scala Native, and Scala.js.
The two work together: both read and write the W3C traceparent header, so a request that crosses services stays in one trace.
JVM agents
Datadog's Java tracer (dd-java-agent) and the OpenTelemetry Java agent instrument the JVM at load time, with no code changes.
dd-java-agent
Run a Datadog Agent next to the service and start the JVM with the tracer:
java -javaagent:/path/to/dd-java-agent.jar \
-Ddd.service=orders -Ddd.env=prod -Ddd.version=1.2.0 \
-jar orders.jarThe tracer sends traces to the Datadog Agent on localhost:8126. To make sure it reads and writes W3C headers, so that Uni's own spans and services without the agent join the same trace, list tracecontext in its propagation styles:
-Ddd.trace.propagation.style=tracecontext,datadogOpenTelemetry Java agent
The OpenTelemetry agent sends OTLP. A Datadog Agent with its OTLP receiver turned on accepts it:
# datadog.yaml
otlp_config:
receiver:
protocols:
http:
endpoint: 0.0.0.0:4318java -javaagent:/path/to/opentelemetry-javaagent.jar \
-Dotel.service.name=orders \
-Dotel.exporter.otlp.endpoint=http://localhost:4318 \
-jar orders.jarThe same OTLP stream also works with an OpenTelemetry Collector, Grafana Tempo, Honeycomb, and other OTLP backends.
What an agent sees in a Uni service
| Part of Uni | Seen by a JVM agent? |
|---|---|
uni-netty server | Yes, as Netty HTTP spans named by method and path |
HTTP client on the JVM (java.net.http) | Yes |
| JDBC, AWS SDK, and other libraries the agent supports | Yes |
The built-in socket server (Http.server on the JVM without uni-netty) | No |
RPC method names (UserService/getUser) | No: they show up as POST /UserService/getUser |
| Scala Native and Scala.js services | No: there is no agent for them |
Uni's built-in tracing covers the rows marked No.
Built-in tracing
A Telemetry instance starts spans and hands them to a SpanExporter. Without an exporter it is disabled, and the server filter and HTTP client skip it at no cost.
import wvlet.uni.http.{Http, TelemetryFilter}
import wvlet.uni.telemetry.{SpanExporter, Telemetry}
val telemetry = Telemetry()
.withServiceName("orders")
.withExporter(SpanExporter.log())Server spans
Add TelemetryFilter to the server as the outermost filter. Each request becomes a server span, and a request with a traceparent header continues the caller's trace:
import wvlet.uni.http.rpc.{RPCHandler, RPCRouter}
trait OrderApi:
def getOrder(id: Int): String
class OrderApiImpl extends OrderApi:
def getOrder(id: Int): String = s"order-${id}"
val server = Http
.server
.withPort(8080)
.withFilter(TelemetryFilter(telemetry))
.withRxHandler(RPCHandler(RPCRouter.of[OrderApi](OrderApiImpl())))How a span is named:
- An RPC call is named after its method, e.g.
OrderApi/getOrder, withrpc.system,rpc.service, andrpc.methodattributes. - An
@Endpointmethod is named after its route template, e.g.GET /orders/:id, with anhttp.routeattribute. - Any other request is named after its HTTP method.
Attributes follow the OpenTelemetry HTTP and RPC semantic conventions:
http.request.method,url.path,server.address, andhttp.response.status_code.- A 5xx response marks the span as failed.
- The query string and RPC arguments are never recorded, since they often carry tokens or personal data.
Client spans
withTelemetry on an HTTP client traces each call. Every call it makes sends a traceparent header:
val client = Http.client.withTelemetry(telemetry).newSyncClientHow client spans work:
- One call is one span. Retries and redirects are recorded as
http.request.resendevents, and the span takes the final response's status. - A call made while a server span is running becomes that span's child, so a request through several services reads as one trace.
url.fullleaves out the query string and any user info in the URL.
Spans in your own code
Telemetry.trace wraps a block in a span. It records an exception and rethrows it:
import wvlet.uni.telemetry.Telemetry
def reindex(): Int = telemetry.trace("reindex") { span =>
span.setAttribute("index.shards", 4)
42
}Inside a traced request, Telemetry.currentSpan returns the server span, and so does the request's TelemetryFilter.CurrentSpan attribute.
Limits
- The current span lives on the thread. It is set while a handler runs its synchronous part, which covers RPC and
@Endpointmethods that return a plain value. Work that continues in anRxcallback on another thread doesn't see it. Pass the span on explicitly there, or wrap the code inTelemetry.withSpan(span) { ... }. - WebSocket upgrades don't run server filters, so WebSocket sessions get no span.
- A JVM agent and Uni share trace ids, not parent links. They don't see each other's current span. The
traceparentheader still keeps both sets of spans in one trace, but some spans may show as siblings instead of parent and child. SpanExporter.log()andSpanExporter.inMemory()are the built-in exporters. To send spans elsewhere, implementSpanExporter.exportSpan, which runs on the thread that ended the span. An exporter that does I/O should buffer spans and send them from its own thread.