Sam Owens is Head of Architecture at Form3. He guides the technical decisions that our engineers make across the platform. Generally, his role involves coordinating decision making. At Form3, we take a collaborative approach to system design and Sam is in charge of facilitating decision discussions.

We spend a lot of time focusing on automation, so we don't have dedicated QA teams. Instead, it is the responsibility of the teams that build the software to ensure that it is well tested and production ready. We run our teams using the DevOps model.

We really believe that focusing on end-to-end tests is the key to that. We put a lot of onus on allowing people to write tests that cover as much of their software in a black box fashion, without relying on unit tests that mock out different behaviour. This allows us to have a very high quality test suite at the end of it.

We have a variety of functional tests:

1. End-to-end tests at the service level: We spin up the infrastructure required for a the service under test using Docker and then we use Pact, which implements consumer contract testing for our internal dependencies.
2. End-to-end platform tests: run full flows for payments through our system. We have fewer of these as they require a fully working platform.
3. Manual smoke tests: the final barrier to ensuring that everything works as expected before we push changes to production.

On the non-functional testing side, we run load tests on our platforms. We also run continuous load on our development environments to ensure that our systems are able to scale correctly.

In general, we don't have many unit tests of our codebases, as we have found low level unit tests (for example ones around a single function) are too tied to the implementation details of the code. Re-implementing services then requires refactoring the tests (sometimes even deleting them!), which can be cumbersome, time consuming and also means it is hard to be confident in the new implementation when you have to change the test too. This allows us to swap underlying services and dependencies without having to rewrite a whole bunch of unit tests.

For us at Form3, end to end tests using Pact for contract testing hits the sweet spot between speed of delivery and test coverage.

Because they are in charge of the full development lifecycle, our engineers have more ownership of our services, which has proven a very successful approach for us. In the 5 years that we've been running our platform, we've not had major incidents caused by defects in production.

Writing the majority of our tests as end-to-end tests can be more complex, especially as our architecture involves asynchronous processing. Brittle or flaky tests are frustrating for engineers.

Fixing timing issues with sleeps and waits or depending on log messages to synchronise your tests are common traps. These quick fixes should be avoided.

Pact is an implementation of consumer contract testing. During test suite definition, we specify the interaction between the consumer and the service. The test suite will specify the URL, parameter and expected responses. These are put in a contract that specifies all of these things together.

Pact then provides a mock server that verifies requests and responses together. The test runner is able to ensure that the server behaviour is as expected both on the consumer and server side. This allows us to test both sides of this contract.

Even though our services are written in Go, we write all of our tests in BDD style, as opposed to table testing.

The choice to write our tests in BDD gives us higher test readability, which is important for our engineers as maintainers.