GopherCon UK 2021:
How Go powered our use of Elasticsearch to increase the performance of our APIs

We had an interesting scaling problem with a Postgres bottleneck. We determined that Elasticsearch would help us to remove this bottleneck. However, in order to do this we needed a solution to listen to the events from our messaging queues and to create and update the relevant documents in Elasticsearch in order to make this work. Enter Go!

Whistle stop tour

This talk shares some key points in our journey to introduce a solution in Go featuring the Go elasticsearch client:

  • How we structured the solution to maximise maintainability and ease of rapid development;
  • The strengths of Go that were really compelling to our use case; and
  • Some of the challenges we overcame along the way, and lessons learned.

In terms of key take-aways:

  • Our solution was successful, and Go helped us to produce an efficient implementation with minimal additional complexity
  • The unidirectional architecture allowed us to evolve the solution as we iterated
  • There were some great questions from the community around some of these points - there’s a good Q&A in the video

Written by

Stuart Murray
Stuart Murray Senior Engineer

Stuart is a Senior Engineer at Form3 in the Core Payments Team. Prior to Form3 Stuart was a consultant and developer working across a range of projects in the healthcare, insurtech and stored value payments sectors. Stuart enjoys working deeply with Go but also works with Java, Rust and Python and previously used C#. Stuart is a member of (and sometimes speaker at) several meetups including GoSheffield.

Read more