airframe-fluentd: Fluentd Logger

airframe-fluentd is a logging library for sending metrics to Fluentd or Treasure Data

• Internally it uses Fluency as the fluentd client.

Usage

build.sbt

libraryDependencies += "org.wvlet.airframe" %% "airframe-fluentd" % "(version)"

Sending Data to Fluentd

import wvlet.airframe.fluentd._

// Define a metric class
case class MyMetric(a:Int, b:String) extends TaggedMetric {
  // Used for defining the default tag prefix for this metric.
  // (tagPrefix).(metricTag) will be used as fluentd tag. 
  override def metricTag: String = "my_metric"
}

// Creating a logger to use the local fluentd (host="localhost", port=24224)
// [optional] tagPrefix: common tag prefix for all metrics  
val loggerFactory = Fluentd.client
  .withTagPrefix("data")
  .newFluentdLoggerFactory()
   
// Create a metric logger for MyMetric class
val l = loggerFactory.getTypedLogger[MyMetric]

l.emit(MyMetric(1, "hello"))   // data.my_metric {"a":1, "b":"hello"}
l.emit(MyMetric(2, "fluentd")) // data.my_metric {"a":2, "b":"fluentd"}

// Close the factory (and underlying loggers)
f.close()

Sending Data to Treasure Data

import wvlet.airframe.fluentd._

// Define a metric class
case class MyMetric(a:Int, b:String) extends TaggedMetric {
  // Specify the table name to store this metric
  override def metricTag: String = "my_metric"
}

// Creating a logger to send log data to Treasure Data Stream Import API:
val loggerFactory = Fluentd.client
  .withTagPrefix("(database name to store logs)")
  .newTDLoggerFactory(apikey = "(Your TD API key)")

// Create a metric logger for MyMetric class
val l = loggerFactory.getTypedLogger[MyMetric]

// Metrics will be stored in data.my_mertric table
l.emit(MyMetric(1, "hello"))   // data.my_metric {"a":1, "b":"hello"}
l.emit(MyMetric(2, "fluentd")) // data.my_metric {"a":2, "b":"fluentd"}

// Make sure closing the logger factory to flush out the logged metrics
loggerFactory.close()

Using Non-Typed Logger

val loggerFactory = Flutentd.client.newFluentdLoggerFactory()

val l = loggerFactory.getLogger
l.emit("data.my_metric", Map("a"->1, "b"->"hello"))

Debugging Metrics

For debugging purpose, use fluentd.withConsoleLogging design:

val d = fluentd.withConsoleLogging // Use a console logger instead of sending logs to Fluentd

d.build[MetricLoggerFactory] { f =>
   val l = f.getTypedLogger[MyMetric]
   l.emit(MyMetric(1, "hello"))   // prints data.my_metric: {"a":1, "b":"hello"}
   l.emit(MyMetric(2, "fluentd")) // prints data.my_metric: {"a":2, "b":"fluentd"}
}