Allow direct access to trace and span id for zio-logging's LogFormat (#…

…842) * Allow direct access to trace and span id for zio-logging * Allow direct access to trace and span id for zio-logging * Implement LogFormats for spanId and traceId * Implement LogFormats for spanId and traceId * Implement LogFormats for spanId and traceId * Implement LogFormats for spanId and traceId * Implement LogFormats for spanId and traceId * Implement LogFormats for spanId and traceId PR fixes * Implement LogFormats for spanId and traceId PR fixes * Implement LogFormats for spanId and traceId * Implement LogFormats for spanId and traceId * Implement LogFormats for spanId and traceId * Implement LogFormats for spanId and traceId
zio · May 31, 2024 · a18c1cd · a18c1cd
1 parent d4d2fd3
commit a18c1cd
Show file tree

Hide file tree

Showing 10 changed files with 441 additions and 16 deletions.
diff --git a/README.md b/README.md
@@ -32,6 +32,12 @@ In order to use this library, we need to add the following line in our `build.sb
 libraryDependencies += "dev.zio" %% "zio-opentelemetry" % "<version>"
 ```
 
+If you're also using [ZIO Logging](https://github.com/zio/zio-logging) you can combine OpenTelemetry with ZIO Logging using:
+
+```scala
+libraryDependencies += "dev.zio" %% "zio-opentelemetry-zio-logging" % "<version>"
+```
+
 For using [OpenTracing](https://opentracing.io/) client we should add the following line in our `build.sbt` file:
 
 ```scala
@@ -49,6 +55,7 @@ libraryDependencies += "dev.zio" %% "zio-opencensus" % "<version>"
 You can find examples with full source code and instructions of how to run by following the links:
 - [OpenTelemetry Example](docs/opentelemetry-example.md)
 - [OpenTelemetry Instrumentation Example](docs/opentelemetry-instrumentation-example.md)
+- [OpenTelemetry ZIO Logging Example](docs/opentelemetry-zio-logging.md)
 - [OpenTracing Example](docs/opentracing-example.md)
 
 ## Articles

diff --git a/build.sbt b/build.sbt
@@ -103,7 +103,7 @@ lazy val root =
   project
     .in(file("."))
     .settings(publish / skip := true)
-    .aggregate(opentracing, opentelemetry, opencensus, docs)
+    .aggregate(opentracing, opentelemetry, opencensus, opentelemetryZioLogging, docs)
 
 lazy val opentracing =
   project
@@ -146,6 +146,20 @@ lazy val opencensus = project
   .settings(mimaSettings(failOnProblem = true))
   .settings(unusedCompileDependenciesFilter -= moduleFilter("io.opencensus", "opencensus-impl"))
 
+lazy val opentelemetryZioLogging = project
+  .in(file("opentelemetry-zio-logging"))
+  .settings(enableZIO())
+  .settings(
+    stdModuleSettings(
+      name = Some("zio-opentelemetry-zio-logging"),
+      packageName = Some("zio.telemetry.opentelemetry.zio.logging")
+    )
+  )
+  .settings(libraryDependencies ++= Dependencies.opentelemetryZioLogging)
+  .settings(mimaSettings(failOnProblem = true))
+  .settings(missinglinkIgnoreDestinationPackages += IgnoredPackage("scala.reflect"))
+  .dependsOn(opentelemetry)
+
 lazy val opentracingExample =
   project
     .in(file("opentracing-example"))
@@ -194,9 +208,14 @@ lazy val docs =
       projectName                                := "ZIO Telemetry",
       mainModuleName                             := (opentracing / moduleName).value,
       projectStage                               := ProjectStage.ProductionReady,
-      ScalaUnidoc / unidoc / unidocProjectFilter := inProjects(opentracing, opentelemetry, opencensus),
+      ScalaUnidoc / unidoc / unidocProjectFilter := inProjects(
+        opentracing,
+        opentelemetry,
+        opencensus
+        //  opentelemetryZioLogging TODO: Causes some weird import issues
+      ),
       scalacOptions --= Seq("-Yno-imports", "-Xfatal-warnings")
     )
     .settings(unusedCompileDependenciesFilter -= moduleFilter("org.scalameta", "mdoc"))
-    .dependsOn(opentracing, opentelemetry, opencensus)
+    .dependsOn(opentracing, opentelemetry, opencensus, opentelemetryZioLogging)
     .enablePlugins(WebsitePlugin)
diff --git a/docs/index.md b/docs/index.md
@@ -32,6 +32,12 @@ In order to use this library, we need to add the following line in our `build.sb
 libraryDependencies += "dev.zio" %% "zio-opentelemetry" % "<version>"
 ```
 
+If you're using [ZIO Logging](https://github.com/zio/zio-logging) you can combine OpenTelemetry with ZIO Logging using:
+
+```scala
+libraryDependencies += "dev.zio" %% "zio-opentelemetry-zio-logging" % "<version>"
+```
+
 For using [OpenTracing](https://opentracing.io/) client we should add the following line in our `build.sbt` file:
 
 ```scala
@@ -49,6 +55,7 @@ libraryDependencies += "dev.zio" %% "zio-opencensus" % "<version>"
 You can find examples with full source code and instructions of how to run by following the links:
 - [OpenTelemetry Example](opentelemetry-example.md)
 - [OpenTelemetry Instrumentation Example](opentelemetry-instrumentation-example.md)
+- [OpenTelemetry ZIO Logging Example](opentelemetry-zio-logging.md)
 - [OpenTracing Example](opentracing-example.md)
 
 ## Articles

diff --git a/docs/opentelemetry-zio-logging.md b/docs/opentelemetry-zio-logging.md
@@ -0,0 +1,144 @@
+---
+id: opentelemetry-zio-logging
+title: "OpenTelemetry ZIO Logging"
+---
+
+`zio-opentelemetry` logging facilities are implemented around OpenTelemetry Logging.
+
+In order to use `zio-opentelemetry` feature with `zio-logging` you should use `zio-opentelemetry-zio-logging` module.
+
+`OpenTelemetry ZIO Logging` contains utilities for combining ZIO Opentelemetry with ZIO Logging
+
+## Installation
+
+```scala
+"dev.zio" %% "zio-opentelemetry-zio-logging" % "<version>"
+```
+
+## Features
+
+### Log formats
+
+This library implements [Log Format](https://zio.dev/zio-logging/formatting-log-records) for span information (`spanId` and `traceId`). 
+To use them you need a `LogFormats` service in the environment. For this, use the `ZioLogging.logFormats` layer which in turn required a suitable `ContextStorage` implementation.
+
+```scala
+//> using scala "2.13.14"
+//> using dep dev.zio::zio:2.1.1
+//> using dep dev.zio::zio-opentelemetry:3.0.0-RC24
+//> using dep dev.zio::zio-opentelemetry-zio-logging:3.0.0-RC24
+//> using dep io.opentelemetry:opentelemetry-sdk:1.38.0
+//> using dep io.opentelemetry:opentelemetry-sdk-trace:1.38.0
+//> using dep io.opentelemetry:opentelemetry-exporter-logging-otlp:1.38.0
+//> using dep io.opentelemetry.semconv:opentelemetry-semconv:1.22.0-alpha
+
+import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingSpanExporter
+import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingLogRecordExporter
+import io.opentelemetry.api.common.Attributes
+import io.opentelemetry.sdk.trace.SdkTracerProvider
+import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor
+import io.opentelemetry.sdk.logs.SdkLoggerProvider
+import io.opentelemetry.sdk.logs.`export`.SimpleLogRecordProcessor
+import io.opentelemetry.sdk.resources.Resource
+import io.opentelemetry.semconv.ResourceAttributes
+import io.opentelemetry.sdk.OpenTelemetrySdk
+import io.opentelemetry.api
+import zio._
+import zio.logging.console
+import zio.logging.LogFormat._
+import zio.telemetry.opentelemetry.tracing.Tracing
+import zio.telemetry.opentelemetry.OpenTelemetry
+import zio.telemetry.opentelemetry.context.ContextStorage
+import zio.telemetry.opentelemetry.zio.logging.LogFormats
+import zio.telemetry.opentelemetry.zio.logging.ZioLogging
+
+object ZioLoggingApp extends ZIOAppDefault {
+
+  val instrumentationScopeName = "dev.zio.LoggingApp"
+  val resourceName             = "logging-app"
+
+  // Prints to stdout in OTLP Json format
+  val stdoutLoggerProvider: RIO[Scope, SdkLoggerProvider] =
+    for {
+      logRecordExporter  <- ZIO.fromAutoCloseable(ZIO.succeed(OtlpJsonLoggingLogRecordExporter.create()))
+      logRecordProcessor <- ZIO.fromAutoCloseable(ZIO.succeed(SimpleLogRecordProcessor.create(logRecordExporter)))
+      loggerProvider     <-
+        ZIO.fromAutoCloseable(
+          ZIO.succeed(
+            SdkLoggerProvider
+              .builder()
+              .setResource(Resource.create(Attributes.of(ResourceAttributes.SERVICE_NAME, resourceName)))
+              .addLogRecordProcessor(logRecordProcessor)
+              .build()
+          )
+        )
+    } yield loggerProvider
+
+  // Prints to stdout in OTLP Json format
+  val stdoutTracerProvider: RIO[Scope, SdkTracerProvider] =
+    for {
+      spanExporter   <- ZIO.fromAutoCloseable(ZIO.succeed(OtlpJsonLoggingSpanExporter.create()))
+      spanProcessor  <- ZIO.fromAutoCloseable(ZIO.succeed(SimpleSpanProcessor.create(spanExporter)))
+      tracerProvider <-
+        ZIO.fromAutoCloseable(
+          ZIO.succeed(
+            SdkTracerProvider
+              .builder()
+              .setResource(Resource.create(Attributes.of(ResourceAttributes.SERVICE_NAME, resourceName)))
+              .addSpanProcessor(spanProcessor)
+              .build()
+          )
+        )
+    } yield tracerProvider
+
+  val otelSdkLayer: TaskLayer[api.OpenTelemetry] =
+    OpenTelemetry.custom(
+      for {
+        tracerProvider <- stdoutTracerProvider
+        loggerProvider <- stdoutLoggerProvider
+        sdk            <- ZIO.fromAutoCloseable(
+          ZIO.succeed(
+            OpenTelemetrySdk
+              .builder()
+              .setTracerProvider(tracerProvider)
+              .setLoggerProvider(loggerProvider)
+              .build()
+          )
+        )
+      } yield sdk
+    )
+
+  // Setup zio-logging with spanId and traceId labels
+  val loggingLayer: URLayer[LogFormats, Unit] = ZLayer {
+    for {
+      logFormats <- ZIO.service[LogFormats]
+      format = timestamp.fixed(32) |-| level |-| label("message", quoted(line)) |-| logFormats.spanIdLabel |-| logFormats.traceIdLabel
+      myConsoleLogger = console(format.highlight)
+    } yield Runtime.removeDefaultLoggers >>> myConsoleLogger
+  }.flatten
+
+
+  override def run =
+    ZIO
+      .serviceWithZIO[Tracing] { tracing =>
+        val logic = for {
+          // Read user input
+          message <- Console.readLine
+          // Print span and trace ids along with message
+          _ <- ZIO.logInfo(s"User message: $message")
+        } yield ()
+
+        // All log messages produced by `logic` will be correlated with a "root_span" automatically
+        logic @@ tracing.aspects.root("root_span")
+      }
+      .provide(
+        otelSdkLayer,
+        OpenTelemetry.logging(instrumentationScopeName),
+        OpenTelemetry.tracing(instrumentationScopeName),
+        OpenTelemetry.contextZIO,
+        ZioLogging.logFormats,
+        loggingLayer
+      )
+
+}
+```
diff --git a/docs/sidebars.js b/docs/sidebars.js
@@ -10,6 +10,7 @@ const sidebars = {
         "opentracing-example",
         "opencensus",
         "opentelemetry",
+        "opentelemetry-zio-logging",
         "opentelemetry-example",
         "opentelemetry-instrumentation-example"
       ]

diff --git a/...metry-zio-logging/src/main/scala/zio/telemetry/opentelemetry/zio/logging/LogFormats.scala b/...metry-zio-logging/src/main/scala/zio/telemetry/opentelemetry/zio/logging/LogFormats.scala
@@ -0,0 +1,27 @@
+package zio.telemetry.opentelemetry.zio.logging
+
+import zio.logging.LogFormat
+import zio.logging.LogFormat.label
+
+trait LogFormats {
+
+  /**
+   * Will print traceId from current span or nothing when not in span
+   */
+  def traceId: LogFormat
+
+  /**
+   * Will print spanId from current span or nothing when not in span
+   */
+  def spanId: LogFormat
+
+  /**
+   * Label with `traceId` key and [[traceId]] value
+   */
+  def traceIdLabel: LogFormat = label("traceId", traceId)
+
+  /**
+   * Label with `spanId` key and [[spanId]] value
+   */
+  def spanIdLabel: LogFormat = label("spanId", spanId)
+}
diff --git a/...metry-zio-logging/src/main/scala/zio/telemetry/opentelemetry/zio/logging/ZioLogging.scala b/...metry-zio-logging/src/main/scala/zio/telemetry/opentelemetry/zio/logging/ZioLogging.scala
@@ -0,0 +1,36 @@
+package zio.telemetry.opentelemetry.zio.logging
+
+import io.opentelemetry.api.trace.{Span, SpanContext}
+import io.opentelemetry.context.Context
+import zio._
+import zio.logging.LogFormat
+import zio.telemetry.opentelemetry.context.ContextStorage
+
+object ZioLogging {
+
+  def logFormats: ZLayer[ContextStorage, Nothing, LogFormats] = ZLayer {
+    for {
+      ctxStorage <- ZIO.service[ContextStorage]
+    } yield new LogFormats {
+      override def traceId: LogFormat = LogFormat.make { (builder, _, _, _, _, _, fiberRefs, _, _) =>
+        getSpanContext(ctxStorage, fiberRefs).map(_.getTraceId).fold(())(builder.appendText(_))
+      }
+
+      override def spanId: LogFormat = LogFormat.make { (builder, _, _, _, _, _, fiberRefs, _, _) =>
+        getSpanContext(ctxStorage, fiberRefs).map(_.getSpanId).fold(())(builder.appendText(_))
+      }
+
+      private def getSpanContext(ctxStorage: ContextStorage, fiberRefs: FiberRefs): Option[SpanContext] = {
+        val maybeOtelContext = ctxStorage match {
+          case ref: ContextStorage.ZIOFiberRef => fiberRefs.get(ref.ref)
+          case ContextStorage.Native           => Some(Context.current())
+        }
+
+        maybeOtelContext
+          .map(Span.fromContext)
+          .map(_.getSpanContext)
+      }
+    }
+  }
+
+}
diff --git a/...ging/src/test/scala/zio/telemetry/opentelemetry/zio/logging/TelemetryLogFormatsSpec.scala b/...ging/src/test/scala/zio/telemetry/opentelemetry/zio/logging/TelemetryLogFormatsSpec.scala
@@ -0,0 +1,59 @@
+package zio.telemetry.opentelemetry.zio.logging
+
+import io.opentelemetry.api.trace.Tracer
+import io.opentelemetry.sdk.testing.exporter.InMemorySpanExporter
+import io.opentelemetry.sdk.trace.SdkTracerProvider
+import io.opentelemetry.sdk.trace.data.SpanData
+import io.opentelemetry.sdk.trace.`export`.SimpleSpanProcessor
+import zio.Runtime.removeDefaultLoggers
+import zio.telemetry.opentelemetry.context.ContextStorage
+import zio.telemetry.opentelemetry.tracing.Tracing
+import zio.test.{Spec, TestEnvironment, ZIOSpecDefault, assertTrue}
+import zio.{Scope, UIO, ULayer, URLayer, ZEnvironment, ZIO, ZLayer}
+
+import scala.collection.mutable
+import scala.jdk.CollectionConverters._
+
+object TelemetryLogFormatsSpec extends ZIOSpecDefault {
+
+  val inMemoryTracer: UIO[(InMemorySpanExporter, Tracer)] = for {
+    spanExporter   <- ZIO.succeed(InMemorySpanExporter.create())
+    spanProcessor  <- ZIO.succeed(SimpleSpanProcessor.create(spanExporter))
+    tracerProvider <- ZIO.succeed(SdkTracerProvider.builder().addSpanProcessor(spanProcessor).build())
+    tracer          = tracerProvider.get("TracingTest")
+  } yield (spanExporter, tracer)
+
+  val inMemoryTracerLayer: ULayer[InMemorySpanExporter with Tracer] =
+    ZLayer.fromZIOEnvironment(inMemoryTracer.map { case (inMemorySpanExporter, tracer) =>
+      ZEnvironment(inMemorySpanExporter).add(tracer)
+    })
+
+  def tracingMockLayer(
+    logAnnotated: Boolean = false
+  ): URLayer[ContextStorage, Tracing with InMemorySpanExporter with Tracer] =
+    inMemoryTracerLayer >>> (Tracing.live(logAnnotated) ++ inMemoryTracerLayer)
+
+  def getFinishedSpans: ZIO[InMemorySpanExporter, Nothing, List[SpanData]] =
+    ZIO.serviceWith[InMemorySpanExporter](_.getFinishedSpanItems.asScala.toList)
+
+  override def spec: Spec[TestEnvironment with Scope, Any] =
+    suiteAll("opentelemetry-zio-logging LogFormats") {
+      test("SpanId and traceId are extracted") {
+        ZIO.serviceWithZIO[Tracing] { tracing =>
+          import tracing.aspects._
+          val logs = mutable.Buffer[String]()
+
+          for {
+            logFormats <- ZIO.service[LogFormats]
+            format      = logFormats.spanIdLabel |-| logFormats.traceIdLabel
+            zLogger     = format.toLogger.map(logs.append(_))
+            _          <- zio.ZIO.logInfo("TEST").withLogger(zLogger) @@ span("Span") @@ root("Root")
+            spans      <- getFinishedSpans
+            child       = spans.find(_.getName == "Span").get
+            log         = logs.head
+          } yield assertTrue(log == s"spanId=${child.getSpanId} traceId=${child.getTraceId}")
+        }
+      }
+    }.provide(removeDefaultLoggers, tracingMockLayer(), ContextStorage.fiberRef, ZioLogging.logFormats)
+
+}