Handling Errors in Your HTTP Handlers
Updated: Sunday, November 21, 2021
It’s been a while since my last post, so I thought I would write about a pattern that I’ve come to like for handling errors in HTTP Handlers in Go.
I’ve always enjoyed the simplicity of writing web services in Go. You create your http.HandlerFunc
s, add them to your router or server, and you’re done. There’s no magic.
With this simplicity however also comes with some downsides.
One thing I’ve never liked is how you are forced to handle errors in your http.Handler s and http.HandlerFunc s. Any error handling has to be done in your handler code, since the signatures require you to conform to handling a http.ResponseWriter
and *http.Request
without returning anything.
// http.Handler interface
ServeHTTP(w http.ResponseWriter, r *http.Request)
// http.HandlerFunc type
func(http.ResponseWriter, *http.Request)
This is typically fine for small web services/APIs that don’t have a large surface area, but I’ve found this approach breaks down when building APIs with a lot of functionality. It also makes it very difficult to conform to an API specification that has well defined error codes and responses.
Let me elaborate with a somewhat contrived example.
Level Up With Go
Thanks for reading! While you're here, sign up to receive a free sample chapter of my upcoming guide: Level Up With Go.
No spam. I promise.
Implementing a Docker Registry API Endpoint
Let’s say you want to implement the Docker Registry HTTP API V2 specification. This is the same API that Docker Hub implements which the Docker client communicates with when you do a docker pull
or docker push
.
Luckily for us, Docker did a great job when writing this spec and gives overviews of what your registry needs to implement in order to work with the Docker client. Heck, they even provided the appropriate Error Codes that you should return when something goes wrong, along with a description of when it’s appropriate to return these errors.
Back to our example. For the sake of terseness, let’s see what it would look like to implement a single endpoint for Pulling an Image Manifest, which is one of the first endpoints that the Docker client calls when you do a docker pull
.
The spec defines that this endpoint should resolve requests that match:
GET /v2/<name>/manifests/<reference>
Let’s implement this endpoint in pseudo-code, conforming to the http.HandlerFunc
type.
func (a *API) GetImageManifest(w http.ResponseWriter, *r http.Request) {
// TODO: do your auth here, handle unauthorized errors (1)
// get the `name` and `reference` path variables using your favorite routing library (or stdlib if you are a masochist)
var (
vars = mux.Vars(r)
name = vars["name"]
ref = vars["reference"]
)
// TODO: do some validation on those path variables, return errors if invalid (2/3)
// get the image manifest from your database that matches that name/reference
manifest, err := a.DB.GetManifest(name, ref)
if err != nil {
// TODO: handle errors (4/5/6)
}
// encode your manifest to JSON, assume your manifest type has JSON struct tags added already or implements the json.Marshaller interface
if err := json.NewEncoder(w).Encode(manifest); err != nil {
// TODO: handle error (7)
}
w.Header().Set("Content-Length", contentLength)
w.Header().Set("Docker-Content-Digest", digest)
w.WriteHeader(http.StatusOK)
}
Now that doesn’t look too bad at first glance does it? But lets look a bit deeper and see all of the places that we have TODO
s to handle errors.
What Can Go Wrong?
Here are some of the un-happy paths that we need to handle:
- Authorization/Authentication fails: The user making the request is not authenticated in the case of a private image, or has invalid permissions
- The
name
variable from the request does not pass validation - The
ref
variable from the request does not pass validation - The manifest requested is not found
- The manifest found is somehow invalid
- The DB that you are retrieving the manifest from is not available or has some other error retrieving the results
- Encoding the retrieved manifest to JSON fails
This list doesn’t even include the myriad of other networking related errors that can occur in the lifetime of your request, such as requests being cancelled or timing out.
Remember that for each of these error cases, we need to interpret the error and return the appropriate error response so that the Docker client can determine what to do.
This will lead to a lot of repetitive, boilerplate error handling across all of your http.Handler
s such as:
if err == ErrNameInvalid {
// set correct HTTP Status code
// return `NAME_INVALID` error code in JSON response along with description
}
if err == ErrDigestInvalid {
// set correct HTTP Status code
// return `DIGEST_INVALID` error code in JSON response along with description
}
Not to mention what the error handling will look like for calls to lower level code that can return multiple error types such as:
manifest, err := a.DB.GetManifest(name, ref)
if err != nil {
switch t := err.(type) {
case NotFoundError:
// set 404 HTTP Status code
// return `MANIFEST_UNKNOWN` error code in JSON response along with description
case InvalidManifestError:
// set 400 HTTP Status code
// return `MANIFEST_INVALID` error code in JSON response along with description
case DBTimeoutError:
// set correct HTTP Status Code? Probably 500?
// return some appropriate error code in the JSON response?
case ...
}
}
You’ll have to repeat this error handling code across your application whenever you call out to lower level code that do things like interacting with your database or authenticating requests.
Also, think about what happens when someone adds a new error type that can be returned from this underlying code in the future? You’d have to go update all of your error handling code across your application!
A Different (Better?) Way
What if you didn’t have to do all this repetitive error handling at all?
What if you could delegate this error handling to a single place in your application, that would allow you to more easily test your HTTP handlers as well as reduce the surface area of code that needs to change when new error types pop up?
Let’s change our http.HandlerFunc
s to return an error
:
// blasphemy!
type handlerFunc func(w http.ResponseWriter, r *http.Request) error
Obviously this no longer matches http.HandlerFunc
s signature, so we can’t plug it in directly. However, we can adapt our new handlerFunc
to match that which Go expects in one of two ways:
- We can create a helper func to adapt our
handlerFunc
to ahttp.HandlerFunc
:
func handle(f handlerFunc) http.HandlerFunc {
return http.HandlerFunc(w http.ResponseWriter, r *http.Request) {
if err := f(w, r); err != nil {
// do all your error switching/handling here in one place!
}
}
}
- We can also make our
handlerFunc
implementhttp.Handler
:
func (f handlerFunc) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if err := f(w, r); err != nil {
// do all your error switching/handling here in one place!
}
}
How does this change our API code (boilerplate excluded)?
manifest, err := a.DB.GetManifest(name, ref)
if err != nil {
return err
}
if err := json.NewEncoder(w).Encode(manifest); err != nil {
return err
}
w.Header().Set("Content-Length", contentLength)
w.Header().Set("Docker-Content-Digest", digest)
w.WriteHeader(http.StatusOK)
return nil
Then, when mounting our http.Handler
s in our router we can do:
// create a new `handlerFunc` from `a.GetImageManifest` which implements `http.Handler`
r.Handle(handlerFunc(a.GetImageManifest))
// or, return a `http.HandlerFunc` by calling our `handle` helper func
r.HandleFunc(handle(a.GetImageManifest))
Assumptions/Derivations
Now, this pattern does come with some baked in assumptions that I should probably make explicit:
- If the
error
returned from yourhandlerFunc
is nil, then it’s expected that you already calledw.Write
andw.WriteHeader
in your handler - If the
error
returned from yourhandlerFunc
is non-nil, then it’s expected that you haven’t called either, and will let the error handling code do that for you
Of course you could make your own handlerFunc
with any signature you wanted, and define your own ‘rules’. For example you could return an (status int, err error)
to allow specifying the HTTP status code in the handlers themselves. Or you could even create and return your own type to include any metadata you want.
You can even create different adapter funcs/types to handle areas in your code that have different types of errors they can return.
The point is that you aren’t limited to the http.Handler
or http.HandlerFunc
signatures when implementing your handlers.
Testing
Another major benefit that I’ve found when using this pattern is that it makes testing your handler code much easier when testing error conditions, which IMO are where you should spend the most time unit testing anyways.
Instead of inspecting the status code and body of the returned httptest.ResponseRecorder to assert that your errors were handled correctly, you can simply assert the errors returned from your handler code directly such as:
err := myHandler(w, r)
// using the wonderful https://github.com/stretchr/testify assert library
assert.EqualError(t, err, "unauthorized")
Then you can test your error handling code in isolation as well, allowing you to have more complete test coverage!
Wrap Up
I know that I didn’t invent this pattern, nor is it earth shattering. But I haven’t read much about it’s use in the past, so I thought I would write this post in case someone was looking for an alternative way to handle errors while reducing repetition when writing their HTTP handlers in Go.
Maybe try it out next time you need to write an API?