Validating HTTP JSON Requests in Go

When choosing your how you wish to expose your Go services, the choice between gRPC and HTTP may be a fairly difficult one. gRPC affords you a more performant network transport, whilst the tooling around it is still behind that of your more standard HTTP-based APIs.

In this tutorial, we are going to be looking at how we can leverage the go-playground/validator package in order to improve not only the security of our APIs, but also the UX component.

Validation in gRPC Services

One of the key things that I enjoy about gRPC-service development is the strictness imposed upon consumers of the service through the protobuf definitions. The tooling to get this setup is arguably a little more complex and takes a little longer, but once it is in place, it helps to ensure that consumers are sending you payloads that you expect.

The HTTP Alternative

Thankfully though, there is a way of enforcing some level of strictness over the incoming HTTP JSON requests to your Go services. We can employ a technique known as JSON-request validation within our HTTP services which effectively allows us to ensure that the incoming request passes a series of checks prior to it being processed by our service.

A Simple Example

Let’s for example have a look at a user management system that handles tasks such as provisioning new user accounts. We would typically have an endpoint that looked a little like this for creating new users:

HTTP POST /api/v1/user
Content-Type application/json

{"username": "elliot", "email": "support@tutorialedge.net"}

The code in our transport layer would typically receive the body of this HTTP request and try and Unmarshal it into a struct like so:

type User struct {
    Username string `json:"username"`
    Email string `json:"email"`
}

// server setup code

// PostUser - handles the provisioning of a new user account
func (h *Handler) PostUser(w http.ResponseWriter, r *http.Request) {
    var u User
    // ignoring errors is bad - this is just an example
    _ = json.Unmarshal([]byte(request.Body), &u)

    newUser, err := h.UserService.CreateUser(r.Context(), u)
    // handle any errors 
    // send responses back to the caller over the ResponseWriter 
}

Now this flow may work, however we can improve this and add validation rules within this handler function that will validate that, after we have unmarshalled our request body into our struct, it contains all the fields that we expect.

In fact, not only can we validate that these fields are present if they are required, we can also do additional validation on the contents of these fields and ensure that, for example, the email field is in fact a valid email.

Adding Validation With go-playground/validator

GitHub Package Link - go-playground/validator

Let’s extend the example code we have above to include the email validation that we discussed. Using a package such as the go-playground/validator we can do this with additional tag information on our struct and by calling validate after the point at which we have unmarshalled:

import (
    // other imports
    "github.com/go-playground/validator/v10"
    //
)


type User struct {
    Username string `json:"username" validate:"required"`
    Email string `json:"email" validate:"email,required"`
}

// server setup code

// PostUser - handles the provisioning of a new user account
func (h *Handler) PostUser(w http.ResponseWriter, r *http.Request) {
    var u User
    _ = json.Unmarshal([]byte(request.Body), &u)


    validate := validator.New()
	err := validate.Struct(wh)
	if err != nil {
        // log out this error
        log.Error(err)
        // return a bad request and a helpful error message
        // if you wished, you could concat the validation error into this
        // message to help point your consumer in the right direction.
		http.Error(w, "failed to validate struct", 400)
        return
    }

    newUser, err := h.UserService.CreateUser(r.Context(), u)
}

In this example, we have effectively achieved two things:

1. We have been able to validate that both the username and email fields are present in the unmarshalled u User struct.
2. We have added validation on our email field that attempts to validate that whatever string is present, looks like a valid email.

This effectively validates at the entrypoint to our application that consumers of our API cannot send us data that we have no real way of processing.

Additional Types of Validation

Now, it should be noted that we aren’t limited to just basic validation using this package. The readme for this project has an absolutely phenomenal number of different types of validations baked-in to the package which allow for a huge range of different use-cases.

If you are a network engineer building infrastructure-based services then there are a huge number of different network-based validations such as ipv4, ipv6, cidr, url and uri ready for you to use.

Why Should We Validate?

One of the biggest reasons we should aim to validate the incoming requests at the entry-point of our applications is for additional security.

Using this approach, we can help to protect the inner-workings of our services from malformed data, or potential bad-actors injecting things like SQL injection strings into our service.

It certainly is no silver bullet in this regard and you will likely still have to bake in further security to your app, but it is certainly a great start.

How does this improve User Experience?

When developing APIs, the user experience for those who will be calling our APIs can be very important. This is especially the case if you are developing public APIs.

By adding this validation to our transport layer, we can immediately return a 400 Bad Request status to the consumers of our APIs and inform them of what fields are missing, or potentially what fields are not passing advanced validation.

This allows consumers of the API to immediately understand why a given request is failing and then take steps to fix any issues. This is a far better user experience for those as opposed to returning something uninformative like an internal server error or something similar.

More Complex Validation Examples

It should be noted that, for more complex validation use-cases, we are not limited to the list of validators already baked into that package.

If you require more complex validation, then there is the option of writing custom validators and using them in much the same way we do with the baked in variety.

Let’s take a look at the example given in the repo. It shows how we can define a fairly arbitrary example, but it demonstrates that we have full flexibility over what each of these validators actually checks.

   
package validators

import (
	"reflect"
	"strings"

	"github.com/go-playground/validator/v10"
)

// NotBlank is the validation function for validating if the current field
// has a value or length greater than zero, or is not a space only string.
func NotBlank(fl validator.FieldLevel) bool {
	field := fl.Field()

	switch field.Kind() {
	case reflect.String:
		return len(strings.TrimSpace(field.String())) > 0
	case reflect.Chan, reflect.Map, reflect.Slice, reflect.Array:
		return field.Len() > 0
	case reflect.Ptr, reflect.Interface, reflect.Func:
		return !field.IsNil()
	default:
		return field.IsValid() && field.Interface() != reflect.Zero(field.Type()).Interface()
	}
}

The test file for this example shows how we can then register this validator:

v := validator.New()
err := v.RegisterValidation("notblank", NotBlank)

Conclusion

So, in this article, we discussed how you can implement HTTP JSON request validation within your application using the go-playground/validator package. We’ve discussed some of the key benefits and how it can ultimately help to improve the security of your systems.

If you are interested in seeing this validation in action in code that is more indicative of a real-world application then I recommend checking out my latest course - Building Production Ready Services in Go - 2nd Edition