Let’s imagine you are building an API client in Go to make it easier for people to interact with your public REST API. Everything is going great. You’ve got authentication, pagination and awesome error handling in place. One thing that’s still unresolved though is how do you test it?

Your client exists to make HTTP requests and then unmarshal that response data (presumably JSON) into objects that make it easy for your consumers to work with. This means at some point, you’re going to have to actually make ‘real’ HTTP requests in order to test your client.

You have two options:

1. Point your client at your real API
2. Fake it

The benefit of #1 is that you are working with real data. You are hitting your actual endpoints and producing/consuming real JSON. This means that your client can always be guaranteed to work in the real world.

The downsides however include:

1. You must always have internet connectivity in order to you run your tests
2. If you want to test creating/updating/deleting data from your client, then you are actually modifying that data in production
3. Your environment must always be left in a ‘known good’ state

Yes, you could remedy the second issue by having a test/staging environment that mimics your production setup. But this is still another environment that you must maintain and keep in a consistent state.

This is why I’m suggesting that you should..

Fake It

What I mean by faking it is that instead of pointing your client at your real API, you instead point it at a mock implementation that you create in your tests. This allows you test all kinds of situations such as:

1. How your client makes the request and returns the response objects for successful requests
2. How your client handles it if the data returned is malformed or does not exist
3. How your client reacts when server errors occur
4. Pretty much anything you can think of that can happen within an HTTP request/response cycle

Go actually makes it pretty easy to do this type of testing, so I’ll share some tips that I’ve learned that will hopefully help save you some time.

1. Make Your Client Configurable

This may go without saying, but in order to point your client to your local implementation of your API in your tests, you first need to be able to configure the URL that your API resides at.

Let’s say you have a Client type that will be responsible for actually making the calls to your API:

const apiURL = "https://api.github.com/"

// Client holds information necessary to make a request to your API
type Client struct {
    baseURL        string
    httpClient     *http.Client
}

// New creates a new API client
func New() (*Client, error) {
    return &Client{
        baseURL:  apiURL,
            httpClient: &http.Client{
            Timeout: time.Second * 30,
        },
    }, nil
}

In the New function we’ve defined the baseURL of the client to be "https://api.github.com/". This is the default where your production API resides.

Now how do we make this configurable for testing?

There’s a great pattern called functional options that comes in handy here.

// Option is a functional option for configuring the API client
type Option func(*Client) error

// BaseURL allows overriding of API client baseURL for testing
func BaseURL(baseURL string) Option {
    return func(c *Client) error {
        c.baseURL = baseURL
        return nil
    }
}

// parseOptions parses the supplied options functions and returns a configured
// *Client instance
func (c *Client) parseOptions(opts ...Option) error {
    // Range over each options function and apply it to our API type to
    // configure it. Options functions are applied in order, with any
    // conflicting options overriding earlier calls.
    for _, option := range opts {
        err := option(c)
        if err != nil {
            return err
        }
    }

    return nil
}

const apiURL = "https://api.github.com/"

// Client holds information necessary to make a request to your API
type Client struct {
    baseURL        string
    httpClient     *http.Client
}

// New creates a new API client
func New(opts ...Option) (*Client, error) {
    client := &Client{
        baseURL:  apiURL,
            httpClient: &http.Client{
            Timeout: time.Second * 30,
        },
    }

    if err := client.parseOptions(opts...); err != nil {
        return nil, err
    }

    return client, nil
}

This above example modifies our original client to be able to be provided Option funcs for how it is configured.

To change the baseURL when constructing the client, you can now do New(BaseURL("http://localhost:8080/api")) to set the baseURL to whatever you want.

2. Spin Up a Server Using HTTPTest

The Go stdlib contains a great package httptest to well, aid in your testing of HTTP.

You can use it in your tests to create your mock API and do anything you like:

package api

var (
    mux    *http.ServeMux
    server *httptest.Server
    client *Client
)

func setup() func() {
    mux = http.NewServeMux()
    server = httptest.NewServer(mux)

    client, _ = New(BaseURL(server.URL))

    return func() {
        server.Close()
    }
}

Here we create the package scoped variables mux, server and client so that we can have access to them in all of our tests. We also create an instance of httptest.Server and bind it to our mux. We’ll use the mux later to add Handlers.

Another thing to note is the definition of the setup function. This function does the work of creating the server and an instance of our client and also returns a function that is used to ‘teardown’ our server when we’re done with it. This is so each test can be completely independent of one another like so:

func TestListRepos(t *testing.T) {
    teardown := setup()
    defer teardown()

    mux.HandleFunc("/orgs/octokit/repos", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        w.WriteHeader(http.StatusOK)
            // ... return the JSON
    })

    repos, err := client.ListRepos("octokit")
    if err := nil {
        t.Fatal(err)
    }
    // ... other tests here
}

func TestGetRepo(t *testing.T) {
    teardown := setup()
    defer teardown()

    mux.HandleFunc("/repos/octokit/octokit.rb", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        w.WriteHeader(http.StatusOK)
            // ... return the JSON
    })

    repos, err := client.GetRepo("octokit", "octokit.rb")
    if err := nil {
        t.Fatal(err)
    }
    // ... other tests here
}

Now the server is shutdown at the end of each test with the use of the defer call.

3. Use Fixture Data

Perhaps a little known feature of the Go test tool is that it ignores any directory named testdata, which allows you to put anything in there you want without it treating it as a package.

This is the perfect place to put JSON files to return in the responses of your mock API:

func fixture(path string) string {
    b, err := ioutil.ReadFile("testdata/fixtures/" + path)
    if err != nil {
        panic(err)
    }
    return string(b)
}

func TestListRepos(t *testing.T) {
    teardown := setup()
    defer teardown()

    mux.HandleFunc("/orgs/octokit/repos", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        w.WriteHeader(http.StatusOK)
        fmt.Fprint(w, fixture("repos/octokit.json"))
    })

    repos, err := client.ListRepos("octokit")
    if err := nil {
        t.Fatal(err)
    }
    // ... other tests here
}

Simply define a helper function fixture to load and return this JSON data given a path, and you can now return custom data for each of your tests in your HTTP response bodies.

4. Use a _test Package

Related to the above tip, I like to also put my client tests in their own _test package when possible. This allows you to test your API client as your consumers would, externally.

This is not a hard and fast rule, and sometimes you’ll want to test internal functionality which means you can’t put all of your tests in their own package, however I find that it helps me think about my client from a different perspective.

It also helps you find any usability issues that your consumers might find when actually using your library.

Conclusion

Hopefully you’ve learned something that you can use when building and testing your API client libraries in Go. In my opinion, the Go stdlib does a great job of providing just the right amount of primitives to allow you to write great HTTP based tests.