GopherCon UK 2021: Using NATS for multi-cloud event streaming

NATS is a lightweight, easy to deploy, high performance messaging system that builds upon cloud native infrastructure concepts.
It is also open source and written in Go. Form3 has been successfully using NATS for event streaming on our multi cloud architecture, which powers payments for Financial Institutions at high volumes.
This talk shares our knowledge and usage of this cool technology with the tech community.

Whistle stop tour

This talk has multiple sections, building from NATS fundamentals, continuing on to JetStream and then on to how we use NATS in production at Form3. Here are some key highlights of the talk:

  1. Form3’s platform is critical payments infrastructure, processing millions of payments every day across the UK and Europe.
    Our old architecture relied on message fanout using SNS & SQS to send payments to our validation and gateway services, which do all the securing and mapping of the payments against the external payments infrastructure.
  2. While this architecture was easy to start with, scale and secure, we measured a few spikes in latency of 300ms+ when our platform is processing high volumes. It was time to evolve our architecture and NATS was identified as a good solution.
  3. Core NATS is simple, fast and at its core is a fire-and-forget, at-most-once messaging system. You can see our demo implementations of a NATS Core Publisher and NATS Core Subscriber.
  4. JetStream is the recommended option for persistence and message guarantees. It has many features which were required of mature production systems, such as at-least-once delivery, data at rest encryption, horizontal scalability. You can see our demo implementations of a JetStream Publisher, JetStream Push Consumer and JetStream Pull Consumer.
  5. We’ve been using NATS Streaming as an event bus in our data centers in production and successfully delivering over a million messages every day.We are now going to introduce JetStream elsewhere in our payments platform. The usecase of the SNS & SQS fanout fits the NATS usage very well, so we can use JetStream as our event bus. The NATS streaming and JetStream clusters are bridged using NATS leaf nodes.

Written by

github-icongithub-icongithub-icon
Adelina Simion Technology Evangelist

Adelina is a polyglot engineer and developer relations professional, with a decade of technical experience at multiple startups in London. She started her career as a Java backend engineer, converted later to Go, and then transitioned to a full-time developer relations role. She has published multiple online courses about Go on the LinkedIn Learning platform, helping thousands of developers up-skill with Go. She has a passion for public speaking, having presented on cloud architectures at major European conferences. Adelina holds an MSc. Mathematical Modelling and Computing degree.