Sam Owens joins us to tell us all about our approach to testing at Form3. He gives us an overview of our testing strategy, the different types of tests we run and explains how to use Pact for testing your services. Finally, he tells us why he prefers BDD style tests.
Sam Owens is Head of Architecture at Form3. He guides the technical decisions that our engineers make across the platform. Generally, his role involves coordinating decision making. At Form3, we take a collaborative approach to system design and Sam is in charge of facilitating decision discussions.
We spend a lot of time focusing on automation, so we don't have dedicated QA teams. Instead, it is the responsibility of the teams that build the software to ensure that it is well tested and production ready. We run our teams using the DevOps model.
We really believe that focusing on end-to-end tests is the key to that. We put a lot of onus on allowing people to write tests that cover as much of their software in a black box fashion, without relying on unit tests that mock out different behaviour. This allows us to have a very high quality test suite at the end of it.
We have a variety of functional tests:
On the non-functional testing side, we run load tests on our platforms. We also run continuous load on our development environments to ensure that our systems are able to scale correctly.
In general, we don't have many unit tests of our codebases, as we have found low level unit tests (for example ones around a single function) are too tied to the implementation details of the code. Re-implementing services then requires refactoring the tests (sometimes even deleting them!), which can be cumbersome, time consuming and also means it is hard to be confident in the new implementation when you have to change the test too. This allows us to swap underlying services and dependencies without having to rewrite a whole bunch of unit tests.
For us at Form3, end to end tests using Pact for contract testing hits the sweet spot between speed of delivery and test coverage.
Because they are in charge of the full development lifecycle, our engineers have more ownership of our services, which has proven a very successful approach for us. In the 5 years that we've been running our platform, we've not had major incidents caused by defects in production.
Writing the majority of our tests as end-to-end tests can be more complex, especially as our architecture involves asynchronous processing. Brittle or flaky tests are frustrating for engineers.
Fixing timing issues with sleeps and waits or depending on log messages to synchronise your tests are common traps. These quick fixes should be avoided.
Pact is an implementation of consumer contract testing. During test suite definition, we specify the interaction between the consumer and the service. The test suite will specify the URL, parameter and expected responses. These are put in a contract that specifies all of these things together.
Pact then provides a mock server that verifies requests and responses together. The test runner is able to ensure that the server behaviour is as expected both on the consumer and server side. This allows us to test both sides of this contract.
Even though our services are written in Go, we write all of our tests in BDD style, as opposed to table testing.
The choice to write our tests in BDD gives us higher test readability, which is important for our engineers as maintainers.
blogs · 7 min
Network Address Translation, forward proxies, and reverse proxies, are three common techniques for managing network traffic at scale. This blog will attempt to distil each idea into its simplest form, and write a code example where possible
May 6, 2022
blogs · 8 min
Exposing pools of machines to clients, or routing network traffic via an intermediary, are common techniques in distributed computing, and large networks. Network Address Translation, forward proxies, and reverse proxies, are three common techniques for managing network traffic at scale. However, I've always found each of these topics to be somewhat mystical, and I've never understood the fundamentals of how each technique works. This blog post will attempt to distil each idea into its simplest form, and write a code example where possible.
May 4, 2022
blogs · 5 min
Liam and Owen from Aqua Security join us to share their work on the open source static analysis tool, tfsec. They give us an introduction to infrastructure as code with terraform, then explain what are the common problems they are trying to solve with tfsec. Finally, they tell us all about getting started with tfsec and getting involved with the project.
April 27, 2022