Using ProblemDetails in .NET Core 3.1 with C#

What is Problem Details?

When you are an API developer, you often need to convey errors to the client or consumer of the API.

It is not always a happy path when it comes to using an API. Things do go wrong sometimes.

Networks fail.
Databases encounter issues.
Your application could encounter an unhandled exception due to another service downstream that failed to respond correctly!

When developing distributed applications, one must design for failure, as there are many points of failure. And when there are failures that we cannot immediately rectify, we must be able to let our consumers know what to do.

In order to achieve this, in the simplest possible way, we could just throw an exception or return a status code and error message. But does everyone understand this type of a response?

For a long time, there was no standard that was widely accepted in the industry for conveying errors from an API to a client who could then consume this information to decide how to act subsequently.

In order to standardise this problem for both consumers of APIs and for developers, the Internet Engineering Task Force, decided to make a proposal, RFC7807 - Problem Details.

An example:

   {
    "type": "https://example.com/probs/out-of-credit",
    "title": "You do not have enough credit.",
    "detail": "Your current balance is 30, but that costs 50.",
    "instance": "/account/12345/msgs/abc",
    "balance": 30,
    "accounts": ["/account/12345",
                 "/account/67890"]
   }

The RFC is pretty detailed. But let me try and give you a TLDR;

ProblemDetails TLDR

To summarise, ProblemDetails is a type of an API response that is suitable in case of a problem. It has certain fields that have a very specific meaning, if present.

type : A string that contains the URI to the problem type. Although the specification recommends that the URI provides a human-readable documentation for the problem, though this aspect is completely optional. However, this is considered to be the primary identifier for the problem type.
title : A short human-readable summary of the problem type. And must be the same for any occurrence of the problem, except for localisation. This is purely for descriptive use not to be acted upon.
status : The HTTP status code generated by the server for this instance of the problem. This again, if present in the response, must be the same as the HTTP response code from the server.
detail : A human-readable explanation specific to this instance of the problem. This is to provide information to help the client fix the problem and it is not a stacktrace of what happened on the server. This is not intended to be parsed by a machine.
instance : a URI to the specific occurrence of the problem.

The RFC has defined the ProblemDetails to be extensible. This means those responsding, can add useful information to it in the form of additional properties.

What about using it in .NET?

Yes. We can use this concept of conveying information about errors in any web development framework. The RFC does not care about which language or framework you use. IETF RFCs are about creating standards that can be implemented by different vendors, in the language of their choice.

I am going to talk about using it in .NET Core APIs in this post today. Because over the past six months and probably for the forseeable future, I will be spending majority of my time developing APIs in .NET Core.

.NET Core support for ProblemDetails

Did you know that ASP.NET Core WebAPI comes with in-built support for ProblemDetails.

.NET 5.0 is out already but I have not yet trialled it myself to confirm if the solution that I use for this example is going to work. But I am sure, most of the basics will still say the same.

ProblemDetails out of the box

In case of certain errors like NotFound() or BadRequest(), the API already responds with a ProblemDetails instance. An example for this can be seen below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
 
{
  "type": "https://tools.ietf.org/html/rfc7231#section-6.5.4",
  "title": "Not Found",
  "status": 404,
  "traceId": "|bf32e848-4ebaf4a346dec76f."
}

Response headers
 content-type: application/problem+json; charset=utf-8 
 date: Tue,04 May 2021 21:57:35 GMT 
 server: Kestrel 

You could try it if you have a .NET Core WebApi. You can also spot that the implementation is pretty detailed. The type property points to the URL corresponding to the RFC proposal for the HTTP ERROR in this case 404.

This just works out of the box! So long as you have a controller declared like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
 
    [ApiController]
    [Route("[controller]")]
    public class WeatherForecastController : ControllerBase
    {
      ...
      ...
        [HttpGet("{regionId}")]
        public async Task<ActionResult<WeatherForecast>> Get(string regionId)
        {
            return NotFound();
        }
    }

Notice that I am returning a NotFound response and it magically gets serialized as a ProblemDetails instance.

There was no additional configuration involved for this purpose. All I did was create a new .NET Core WebApi project in Visual Studio.

Handling Exceptions with ProblemDetails responses

In most cases, when there is an exception in your application, you would like to return a helpful response with details on how to avoid the exception or mitigate the chances of a failure.

ASP.NET Core does not by default return all exceptions in the form of ProblemDetails, apart from the ones that I have already mentioned above, like basic error ActionResult responses and some ModelValidationErrors.

If you were to throw a custom exception, this would not be serialised as a problem+json. This is where you have to take control of the exception and make use of the ProblemDetails class.

ProblemDetails as a response for all unhandled exceptions

So let us start with a basic API that has absolutely no concept of error handling. You just created your first ASP .NET Core Web Api project using visual studio.

I created one just now and it created a WeatherForecastController.cs with some code like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
 
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace ProblemDetailsDemoWebApi.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class WeatherForecastController : ControllerBase
    {
        private static readonly string[] Summaries = new[]
        {
            "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
        };

        private readonly ILogger<WeatherForecastController> _logger;

        public WeatherForecastController(ILogger<WeatherForecastController> logger)
        {
            _logger = logger;
        }

        [HttpGet]
        public IEnumerable<WeatherForecast> Get()
        {
            var rng = new Random();
            return Enumerable.Range(1, 5).Select(index => new WeatherForecast
            {
                Date = DateTime.Now.AddDays(index),
                TemperatureC = rng.Next(-20, 55),
                Summary = Summaries[rng.Next(Summaries.Length)]
            })
            .ToArray();
        }
    }
}

Default behaviour - unhandled exception without ProblemDetails

Let us take a look.

I am going to assume that you already know what the Startup.cs file is for. So you will have your Configure(IApplicationBuilder, IWebHostEnvironment) method defined with the conditional use of a DeveloperExceptionPage.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
   if (env.IsDevelopment())
   {
       app.UseDeveloperExceptionPage();
   }
   app.UseHttpsRedirection();
   app.UseRouting();
   app.UseAuthorization();
   app.UseEndpoints(endpoints =>
   {
       endpoints.MapControllers();
   });
}

Let us make the WeatherForecastController throw an exception and see how it works now.

Modify the Get method in the said Controller.

1
2
3
4
5
[HttpGet]
public IEnumerable<WeatherForecast> Get()
{
    throw new Exception("Stormy weather destroyed weather sensors! No ilable.");
}

Run the App and you should be presented with something like:

The default Developer exception page with details of the error

This is great for local development. But you know what, you could really get it all in the form of ProblemDetails even while doing your local development.

Let us give that a shot.

Introducing Hellang.Middleware.ProblemDetails

If you use Nuget Package manager console in visual studio:

Install-Package Hellang.Middleware.ProblemDetails

Configure it now. Add the following changes to your Startup.cs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
public void ConfigureServices(IServiceCollection services)
{
    services.AddProblemDetails();
    services.AddControllers();
}
 // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    app.UseProblemDetails();
    //if (env.IsDevelopment())
    //{
    //    app.UseDeveloperExceptionPage();
    //}

Notice that I have commented out the use of the Developer Exception page. This is for demonstration so that you can clearly see what is being returned.

Run the solution again and let us take a look at what we get without any other change.

GET /weatherforecast HTTP/2
Host: localhost:44327

RESPONSE:
{
    "type": "https://httpstatuses.com/500",
    "title": "Internal Server Error",
    "status": 500,
    "detail": "Stormy weather destroyed weather sensors! No data available.",
    "errors": [
        {
            "message": "Stormy weather destroyed weather sensors! No data available.",
            "type": "System.Exception",
            "raw": "System.Exception: Stormy weather destroyed weather sensors! No data available.\r\n   at ProblemDetailsDemoWebApi.Controllers.WeatherForecastController.Get() in D:\\Code\\ProblemDetailsDemoWebApi\\ProblemDetailsDemoWebApi\\Controllers\\WeatherForecastController.cs:line 29\r\n   at lambda_method(Closure , Object , Object[] )\r\n   at Microsoft.Extensions.Internal.ObjectMethodExecutor.Execute(Object target, Object[] parameters)\r\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)\r\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()\r\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)\r\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)\r\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)\r\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\r\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)\r\n   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)\r\n   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)\r\n   at Hellang.Middleware.ProblemDetails.ProblemDetailsMiddleware.Invoke(HttpContext context)",
            "stackFrames": [
                {
                    "filePath": "D:\\Code\\ProblemDetailsDemoWebApi\\ProblemDetailsDemoWebApi\\Controllers\\WeatherForecastController.cs",
                    "fileName": "WeatherForecastController.cs",
                    "function": "ProblemDetailsDemoWebApi.Controllers.WeatherForecastController.Get()",
                    "line": 29,
                    "preContextLine": 23,
                    "preContextCode": [
                        "            _logger = logger;",
                        "        }",
                        "",
                        "        [HttpGet]",
                        "        public IEnumerable<WeatherForecast> Get()",
                        "        {"
                    ],
                    "contextCode": [
                        "            throw new Exception(\"Stormy weather destroyed weather sensors! No data available.\");"
                    ],
                    "postContextCode": [
                        "            var rng = new Random();",
                        "            return Enumerable.Range(1, 5).Select(index => new WeatherForecast",
                        "            {",
                        "                Date = DateTime.Now.AddDays(index),",
                        "                TemperatureC = rng.Next(-20, 55),",
                        "                Summary = Summaries[rng.Next(Summaries.Length)]"
                    ]
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "lambda_method(Closure , object , object[] )",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.Extensions.Internal.ObjectMethodExecutor.Execute(object target, object[] parameters)",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor+SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, object controller, object[] arguments)",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted)",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted)",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                },
                {
                    "filePath": null,
                    "fileName": null,
                    "function": "Hellang.Middleware.ProblemDetails.ProblemDetailsMiddleware.Invoke(HttpContext context)",
                    "line": null,
                    "preContextLine": null,
                    "preContextCode": null,
                    "contextCode": null,
                    "postContextCode": null
                }
            ]
        }
    ],
    "traceId": "|9d778f7c-4520ebd52c5fd90f."
}

Look at the level of detail in the response. All this for installing a package and writing two lines of code to configure! That my friend is great return on investment!

But that level of detail, is probably a bit too much for our clients

Configuration to fine tune details in response

Of course, you can tweak your response to your taste.

Mr. Hellang, did a great job thinking about the configuration bit. Let us take a stab at that now.

A full list of options that can be configured can be viewed in the github repository of the Hellang.Middleware.ProblemDetails

Let us make a small change to our Configuration to avoid sending stacktraces in Production environment. Make the following change in ConfigureServices() in Startup.cs.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
public void ConfigureServices(IServiceCollection services)
{
    services.AddProblemDetails(opts =>
    {
        // Control when an exception is included
        opts.IncludeExceptionDetails = (ctx, ex) =>
        {
            // Fetch services from HttpContext.RequestServices
            var env = ctx.RequestServices.GetRequiredService<IHostEnvironment>();
            return env.IsDevelopment() || env.IsStaging();
        };
    });

Let us try changing the ASPNETCORE_ENVIRONMENT variable’s value when running the project this time. You can do this by updating the launchSettings.json file and modifying the value that corresponds to the property of that name.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
{
  "$schema": "http://json.schemastore.org/launchsettings.json",
  "iisSettings": {
    "windowsAuthentication": false,
    "anonymousAuthentication": true,
    "iisExpress": {
      "applicationUrl": "http://localhost:19197",
      "sslPort": 44327
    }
  },
  "profiles": {
    "IIS Express": {
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "weatherforecast",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Production"
      }
    },

Now launching the application from Visual Studio, would give you:

{
    "type": "https://httpstatuses.com/500",
    "title": "Internal Server Error",
    "status": 500,
    "traceId": "|7d28a4ed-479221c8689242b4."
}

So in production, you will not receive all those additional stack trace. The moment you switch back to a non-production environment you still get all the details that it was giving you earlier.

Customising ProblemDetails

So far we have just done the bare minimum to get an understanding of what can be achieved using the ProblemDetails middleware.

Let us now take some time to think about how we could use ProblemDetails as our default response mechanism in an application.

Imagine developing the world’s best weather forecast API and several clients use your API to report weather. So you would like to standardize the error responses with sufficient information to lead users to the right course of action.

How would you do it? There are many ways to do this.

You could create your own extension of ProblemDetails in order to convey the right information for a particular instead of relying on generic content. You could add custom properties to your extended ProblemDetails instance to give additional information for that particular use-case.
You could configure ProblemDetails middleware to map certain type of exceptions to certain HTTP status codes
You could also create a custom middleware, maybe to parse exceptions and log into specific stores that would then alert the team responsible for owning it. In this custom middleware, you could then choose to respond with a decent message after the logging, to help the client understand the problem.

The building blocks

Let us create some classes to try a Custom Exception scenario.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
public class StormyWeatherProblemDetails : ProblemDetails
{
    public int LastReportedWindGust { get; set; }
}

public class ProblemDetailsException : Exception
{
    public string Type { get; set; }
    public string Detail { get; set; }
    public string Title { get; set; }
    public string Instance { get; set; }
}

public class StormyWeatherException : ProblemDetailsException
{
    public int LastReportedWindGust { get; set; }
    public StormyWeatherException(string instance, int lastReportedWindGust)
    {
        Type = "https://awesome-weather-forecasts.com/we-got-problems/storm-destruction";
        Title = "Storm destroyed weather sensors.";
        Detail = "Unable to retrieve latest weather forecast due to loss of hardware.";
        LastReportedWindGust = lastReportedWindGust;
        Instance = instance;
    }
}

We now have the following:

A standard base class for Problem Details type exceptions ProblemDetailsException
A custom Exception class that extends the base class - StormyWeatherException
A custom ProblemDetails class - StormyWeatherProblemDetails

These are our building blocks.

The configuration

What we now want to do is:

In case of an error, raise an instance of StormyWeatherException
Map the details from this custom exception to the StormyWeatherProblemDetails to be sent as a response to our client

The simplest way to do this is by configuring the ProblemDetailsMiddleware to map the exception to the corresponding ProblemDetails.

If you use an object mapper in your solution, you could define the mappings in whatever convention appropriate for the mapping library and use that.

In this example, I am going to explicitly write the mapping in the ConfigureServices method.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
services.AddProblemDetails(opts =>
{
    // Control when an exception is included
    opts.IncludeExceptionDetails = (ctx, ex) =>
    {
        // Fetch services from HttpContext.RequestServices
        var env = ctx.RequestServices.GetRequiredService<IHostEnvironment>();
        return env.IsDevelopment() || env.IsStaging();
    };

    opts.Map<StormyWeatherException>((swe) => new StormyWeatherProblemDetail()
    {
        Type = swe.Type,
        Title = swe.Title,
        Detail = swe.Detail,
        Instance = swe.Instance,
        LastReportedWindGust = swe.LastReportedWindGust,
        Status = StatusCodes.Status500InternalServerError
    });
});

Now for the purpose of demonstration, we have to throw this exception from our Controller.

1
2
3
4
5
[HttpGet]
public IEnumerable<WeatherForecast> Get()
{
    throw new StormyWeatherException("/weatherforecast", 101);
}

Run the app again and voila! You now have your exception in ProblemDetails format as below:

{
    "lastReportedWindGust": 101,
    "type": "https://awesome-weather-forecasts.com/we-got-problems/storm-destruction",
    "title": "Storm destroyed weather sensors.",
    "status": 500,
    "detail": "Unable to retrieve latest weather forecast due to loss of hardware.",
    "instance": "/weatherforecast",
    "traceId": "|ff2dbba9-41b908973f06fd2e."
}

So now we have all responses in a standard ProblemDetails format because we are using the error handling given to us by the package.

It is possible that in your project, you probably already have a custom Error Handling pipeline implemented.

We will take a look at how we can integrate ProblemDetails responses in such a solution in another post.

Code

As I had to demonstrate snippets and how things worked, I obviously had to create my own solution. So I have pushed the repo to Problem Details Demo.

Feel free to take a read through the WeatherForecastController, Startup and other ProblemDetails specific files in the project.

What is Problem Details?#

ProblemDetails TLDR#

What about using it in .NET?#

.NET Core support for ProblemDetails#

ProblemDetails out of the box#

Handling Exceptions with ProblemDetails responses#

ProblemDetails as a response for all unhandled exceptions#

Default behaviour - unhandled exception without ProblemDetails#

Introducing Hellang.Middleware.ProblemDetails#

Configuration to fine tune details in response#

Customising ProblemDetails#

The building blocks#

The configuration#

Code#