• Castle.Core
• log4net

This tool allows you to easily capture the response time of any dependent service in your log files without relying on any external tools or deployments. The information is captured directly in your applications log file. If the SLA is breached the entry is logged at ERROR level, otherwise it will be done at DEBUG (this will be configurable in future versions)

An example log entry:

DEBUG Wonga.SLAMonitor.Synchronous.ServiceLevelAgreementMonitor - SLA=CustomerAddressDetailsProvider ResponseTime=53 milliseconds
ERROR Wonga.SLAMonitor.Synchronous.ServiceLevelAgreementMonitor - SLA=CustomerAddressDetailsProvider ResponseTime=2501 milliseconds

In this example we want to monitor the response time of an API which provides a customer's address. We have conveniently wrapped the call to the API in an ICustomerAddressDetailsProvider interface so to request the address for a particular customer you would typically do something like this:

    Guid customerId = .... // Just included for completeness
    ICustomerAddressDetailsProvider addressProvider = new CustomerAddressDetailsProvider();
    addressProvider.GetCurrentAddressDetails(customerId);

Now in order to track the response time we will use DynamicProxy to intercept any calls to the addressProvider and pass them through our ServiceLevelAgreementMonitor. Our code needs to be altered to create the proxy before calling our service. We do that like so:

    Guid customerId = .... // Just included for completeness
    
    var generator = new ProxyGenerator(); // We use Castle.DynamicProxy to create a proxy which will intercept the calls to our actual addressProvider implementation
    IServiceLevelAgreementProvider slaProvider = new SLAProvider(); // Don't worry about this for the moment just know that it is used to determine when a particular service breaches it's SLA
    var concreteAddressProvider = new CustomerAddressDetailsProvider();
    var slaMonitor = new ServiceLevelAgreementMonitor(slaProvider);

    ICustomerAddressDetailsProvider addressProvider = generator.CreateInterfaceProxyWithTarget<ICustomerAddressDetailsProvider>(concreteAddressProvider, slaMonitor); //Proxy calls to the concreteAddressProvider through slaMonitor
    addressProvider.GetCurrentAddressDetails(customerId);  // Call the provider just like we did before

Now all calls to the address provider are wrapped by the SLAMonitor, which is great however we still need to provide a mechanism for the monitor to determine what the SLA for a particular request is. This is what the IServiceLevelAgreementProvider which we inject into the SLAMonitor is for. It has one method which we must provide an implementation for:

   TimeSpan GetServiceLevelAgreement(Type key);

This method is called to determine whether the SLA is breached or not when the request is processed. The type passed in is the type of the actual implementation, so CustomerAddressDetailsProvider in this example (it is not the interface, afterall it is the concrete implementation we are interested in monitoring).

The only requirement is to provide a concrete implementation of this IServiceLevelAgreementProvider, an example is below, with a default value of 1 second, but for our address lookup we are going to have a stricter SLA of half a second. Ordinarily this implementation would simply be a dictionary of values indexed by the Type, but this implementation is just for illustration:

    public class SLAProvider : IServiceLevelAgreementProvider
    {
        public TimeSpan GetServiceLevelAgreement(Type key)
        {
            int slaInMilliseconds = key == typeof(CustomerAddressDetailsProvider) ? 500: 1000;
            return TimeSpan.FromMilliseconds(slaInMilliseconds);
        }
    }

Now whenever you make a call to the provider you will see this in the logs:

DEBUG Wonga.SLAMonitor.Synchronous.ServiceLevelAgreementMonitor - SLA=Wonga.DocumentGeneration.DataProvider.CustomerDetails.CustomerAddressDetailsProvider ResponseTime=5 milliseconds

Or an error when the SLA is breached:

ERROR Wonga.SLAMonitor.Synchronous.ServiceLevelAgreementMonitor - SLA=Wonga.DocumentGeneration.DataProvider.CustomerDetails.CustomerAddressDetailsProvider ResponseTime=2501 milliseconds

As of version 2.0.0 you can override the log levels by passing a LoggingConfiguration to SLAMonitor. It will default to DEBUG/ERROR when the SLA is met/breached

To use the Synchronous monitor you must do the following:

• Provide a concrete implementation of IServiceLevelAgreementProvider
• Create a proxy around the provider you wish to monitor using Castle.DynamicProxy

The asynchronous SLA measurement bases on concept of request-response object pairs, correlated together with GUID identifier.

At the point when request is being made, it's correlation ID is being captured and the time measurement starts. When response is received, it's correlation ID is being used to find out the corresponding request entry. If found, a request-response time is being logged with DEBUG or ERROR level, depending if SLA was violated or not.

It is possible to associate multiple response object types with a specific request, where SLA would be measured only for first response (see example below).

All request-response measurements are being kept in memory and are not persisted.

Changes Since version 2.0.0

1. A QuotaPerMessageType property has been added to SlaProcessor which limits the number of concurrently tracked requests, preventing from memory overuse and potential crashes. The quota request message type based, which means that each request type has own quota. By default it is set to 1000 messages but can be configured to different value in SlaProcessor constructor.
2. A ProcessSlaTimeouts() method has been added to SlaProcessor allowing to detect the request messages which processing time is longer that 2 times the longest SLA defined for this message. While this method can be manually called at any time, SlaProcessor has been extended with timeoutValidationSchedulerFactory as well, allowing to schedule ProcessSlaTimeouts() calls.
3. A SlaProcessorBuilder class has been added to configure and instantiate SlaProcessor in easy and fluent way, allowing to configure processor with default or customized settings.

The solution consists of following classes:

• SlaProcessor capturing and measuring SLA for request-response object pairs,
• SlaProvider providing configured SLA definitions,
• SlaDefinitionBuilder constructing SLA definitions in a fluent manner.
• SlaProcessorBuilder constructing SlaProcessor in a fluent manner.
• SlaTimeoutValidationScheduler allowing to call SlaProcessor.ProcessSlaTimeouts() periodically.

var provider = new SlaProvider();

SlaDefinitionBuilder.AddSla<GenerateInstalmentsSecciDocument, GenerateDocumentResponse>(
    TimeSpan.FromSeconds(25), //SLA
    req => req.RequestId, // how to obtain request correlation ID
    rsp => rsp.RequestId, // how to obtain response correlation ID
    provider);

SlaDefinitionBuilder.For<VerifyInstalmentLoan>(req => req.ApplicationId) // request and its correlation ID
    .AddSla<IInstalmentApplicationAccepted>(TimeSpan.FromSeconds(120), rsp => rsp.ApplicationId) // response
    .AddSla<IInstalmentApplicationDeclined>(TimeSpan.FromSeconds(120), rsp => rsp.ApplicationId) // other response
    .AddSla<IInstalmentApplicationReferred>(TimeSpan.FromSeconds(120), rsp => rsp.ApplicationId) // ...
    .AddSla<IInstalmentApplicationDecisionTimedOut>(TimeSpan.FromSeconds(120), rsp => rsp.ApplicationId)
    .Configure(provider);

var processor = SlaProcessorBuilder.For(provider).Build();

SlaDefinitionBuilder offers two methods for defining SLA.

The first method allows to associate a define request-response pair with a SLA.

The second method allows to associate multiple response types for specific request, where each response type can have different SLA. If this method is used, the SLA would be measured from the occurrence of request with specific correlation ID, till the first occurrence of any response with matching correlation ID.

As of version 2.0.0, SlaProcessor has to be instantiated via SlaProcessorBuilder. The simplest usage is as follows:

var processor = SlaProcessorBuilder.For(provider).Build();

SlaProcessorBuilder allows however to customize processor settings:

• a factory creating timeout validation scheduler, for triggering ProcessSlaTimeouts() (WithTimeoutValidationSchedulerFactory() and WithTimeoutValidationScheduler()),
• logging level configuration for logging SLA being breached/met (WithLoggingConfiguration()),
• quota per message type (WithQuotaPerMessageType()).

The default settings are:

• DefaultQuotaPerMessageType -> 1000,
• DefaultLoggingConfiguration -> ERROR/DEBUG for SLA being met/breached,
• DefaultTimeoutValidationInterval -> 10 seconds,
• DefaultTimeoutValidationSchedulerFactory -> SlaTimeoutValidationScheduler instance, configured with DefaultTimeoutValidationInterval.

Please note that SlaProcessor is disposable and should be disposed after usage (usually IoC containers offers that functionality out of hand).

The easiest integration can be done via message mutator like one below:

public class SlaMonitorLoggingMutator : IMessageMutator
{
    private readonly ISlaProcessor _processor;

    public SlaMonitorLoggingMutator(ISlaProcessor processor)
    {
        _processor = processor;
    }

    public object MutateOutgoing(object message)
    {
        _processor.ProcessOutgoingMessage(message);
        return message;
    }

    public object MutateIncoming(object message)
    {
        _processor.ProcessIncomingMessage(message);
        return message;
    }
}

The SlaMonitorLoggingMutator has to be later registered in EndpointConfiguration like below:

    config.Configurer.ConfigureComponent<SlaMonitorLoggingMutator>(DependencyLifecycle.SingleInstance);

For more details, read NServiceBus docs.