logo-dark

Conf42 Golang 2022: Load testing with F1

Due to a lack of functionality in existing solutions, we wrote our own open source load testing tool, F1, at Form3 to test our asynchronous system. This talk gives you an introduction to performance testing, a comparison of common testing tools and a short introduction of how to use f1 to write your own testing scenarios in Go.

What is performance testing?

Performance testing is the general name for tests that check how the system behaves and performs. The purpose of these tests is to examine stability, scalability and reliability of your software and infrastructure. Generally, performance tests are integration tests that should test your system end-to-end. They live in the middle of the testing pyramid.

Before performance testing, it’s important to determine your system’s business goals, so you can tell if your system behaves satisfactorily or not according to your customer needs. Often, performance testing is done on anticipated future load to see a system’s growth runway. This is important for a high volume payments platform like ours.

Under the umbrella of performance testing, we can look at 3 test subtypes.

Load tests

Load testing tells us how many concurrent requests your system can handle. These tests should be performed all the time in order to ensure your system is behaving correctly, which is why it should be integrated into your continuous integration cycles.

Spike tests

A spike test checks the upper limits of your system by testing it under extreme loads. It includes a sudden, high ramp-up in users and laod.

Soak tests

A soak test checks the sustainability of your system by testing it over a long period of time. It incluse a slow, gradual ramp-up over time.

Common tools

Two of the most common popular tools that are used for test configuration and running areApache JMeter and Grafana K6. Let's have a look at each of them.

Apache JMeter

JMeter is a popular, open source Java testing tool. It allows us to configure tests using a recording GUI and predefined templates. We can use long polling or another request to check whether an asynchronous operation is completed.

JMeter provides thread groups and a ramp up period in seconds load testing distributed systems.

Grafana K6

K6 is another popular load testing tool. It is an open source Go project. Tests are configured using a scripting language similar to JavaScript.

K6 does not provide support for promises or async execution, but we can achieve async testing support using virtual users.

Load for the test is configured using scenarios in an options object, which states how many requests to run for each stage of the test. This allows us to configure linear and step ramp up.

Our ideal load testing tool

At Form3, we invest a lot of engineering time into performance testing our platform. We initially used k6 to develop and run these tests, but it did not fully fit our ideal load testing tool.

Our ideal tool should allow us to:

  1. Easily write asynchronous tests, which integrate with our queues and services. This was not always easy to do in Javascript.
  2. A allow our engineers to write tests in Go, which is what they’re most comfortable in.
  3. Run different modes of load. As our platform operates under huge amounts of load, we want to be able to configure different kinds of load.

F1 example

The existing solutions did not provide us any of our ideal load testing tool features, so this is why we decided to write our own solution and then open source it for the community to use. You can find it under form3tech-oss/f1.

We have also created a demonstration/tutorial repository under form3tech-oss/f1-example. We will briefly discuss this example in this blogpost.

The example docker-compose.yml sets up a simple service which uses goaws local SQS mock. This is the environment we will be running tests on.

Writing tests

Writing tests using f1 is easy - you need to import the library and declare it in the main() function of a new command. The f .Add method registers a new scenario to be run with f1.

import (
	"net/http"
	"time"

	"github.com/form3tech-oss/f1/v2/pkg/f1"
	"github.com/form3tech-oss/f1/v2/pkg/f1/testing"
)

func main() {
	f := f1.New()
	f.Add("testScenario", testScenario)
	f.Execute()
}

After setting up the SQS client and consuming messages from it, we can configure our run function which will be called by thef1test assertions.

  runFn := func(t *testing.T) {
        // Our test iteration code goes here.
        res, err := http.Post("http://localhost:8080/payments", "application/json", nil)
        t.Require().NoError(err)
        t.Require().Equal(http.StatusAccepted, res.StatusCode)
        timer := time.NewTimer(10 * time.Second)
        for {
            select {
            case <-timer.C:
                t.Require().Fail("no message received after timeout")
                return
            case <-messagesChan:
                t.Logger().Info("message received, iteration success")
                return
            }
        }
    }

You can see the entire test scenario configuration on GitHub

Running tests

Once our test is written we can compile the f1 binary to make it easier to run.

go build -o f1 ./cmd/f1/main.go

Then, it's time to start up the example test environment consisting of our payments service and the mock SQS queue.

docker-compose up -d

Finally, the configured scenario is easily run using the f1 CLI. This command will run the test at a constant rate for 10 seconds.

./f1 run constant testScenario -r 1/s -d 10s

The command outputs some basic metrics for the test service.

Conclusions

Writing our tests in Go has been a huge game changer for our engineers, as it allows us to make use of goroutines and channels for test configuration.

f1 is feature complete, we use it every day and you can too. It's freely available on GitHub.

Happy load testing!

Written by

github-icongithub-icongithub-icon
Adelina Simion Technology Evangelist

Adelina is a polyglot engineer and developer relations professional, with a decade of technical experience at multiple startups in London. She started her career as a Java backend engineer, converted later to Go, and then transitioned to a full-time developer relations role. She has published multiple online courses about Go on the LinkedIn Learning platform, helping thousands of developers up-skill with Go. She has a passion for public speaking, having presented on cloud architectures at major European conferences. Adelina holds an MSc. Mathematical Modelling and Computing degree.

Written by

github-icongithub-icon
Andy Kuszyk Staff Engineer

Andy Kuszyk is a Staff Engineer at Form3, based in Southampton. He's been working as a software engineer for 8 years with a variety of technologies, including .NET, Python and most recently Go. Check out more of his tech articles on his blog.