Logging with Serilog and Application Insights

.NET Core provides a simple logging abstraction with most of required logging features used by an application. Everywhere in the application, when logging is required, the Classes generating logs should receive via dependency injection an instance of ILogger<> to use as the logger object. The ILogger<> instances are created by Logging Factory registered by each application and will abstract the logging library from the application logging.

The standard library has limited capabilities on log forwarding and requires additional packages to add proper filtering, enrichment and forwarding. For this reason the use of Serilog is required to extend the default logging features.

Serilog integrates with the logger abstraction provided by DotNet Core. It provides additional Sink(outputs), Filter capabilities and Log Enrichment features to complete the limited set of features in .NET Core default loggers. Every services implemented using .NET Core should make use of Serilog library to standardize the logging approach.

Sinks

By default, the Console and ApplicationInsights sinks should be used.

The console sink will output all logging generated by the service to the runtime console, and will make the logs available via command line tools like kubectl (kubectl get logs) providing a quick and straight forward way of getting the logs for a particular instance.

The Application Insights sink will make sure all logs generated(and filtered) are forwarded for the logging platform for correlation and future investigations.

Application Insights is the chosen logging platform and will aggregated all logs generated by all services. Console sink is an useful secondary output in order to facilitate investigation of issues with particular service that are failing. Having both setup, will make very straightforward to identify issues in real time or delayed investigation.

Serilog Configuration

The following is a sample appsettings.json file with annotations about logging configuration:

{
  "Serilog": {    // Logging configuration for Serilog
    "Using": [
      "Serilog.Sinks.Console",              // Import console sink package
      "Serilog.Sinks.ApplicationInsights"   // Import ai sink package
    ],
    "MinimumLevel": {
      "Default": "Information", // minimal log level for everything
      "Override": {}            // Override for individual log sources
    },
    "WriteTo": [
      { "Name": "Console" },    // Write Logs to console
      {
        "Name": "ApplicationInsights",  // Write Logs to App Insights
        "Args": {
          "telemetryConverter": "Serilog.Sinks.ApplicationInsights.Sinks.ApplicationInsights.TelemetryConverters.TraceTelemetryConverter, Serilog.Sinks.ApplicationInsights"
        }
      }
    ],
    "Enrich": [ "FromLogContext", "WithMachineName", "WithThreadId" ], // Add custom attributes to the logs, Hostname and the ThreadId
    "Destructure": [],
    "Properties": {             // fixed attributes added to every log for filtering purposes
      "Domain": "identity",     // domain name added to every log
      "Service": "identityapi", // service generating the logs
      "Environment": "dev",     // environment this instance belongs to
    },
    "Filter": [                 // Ignore logs that matches the following filters
      {
        "Name": "ByExcluding",
        "Args": {
          "expression": "EndsWith(RequestPath, '/health')" // Filter out health requests to reduce useless log information. Not required for non api services.
        }
      }
    ]
  }
}