GopherCon UK 2021: Using NATS for multi-cloud event streaming
NATS is a lightweight, easy to deploy, high performance messaging system that builds upon cloud native infrastructure concepts.
It is also open source and written in Go. Form3 has been successfully using NATS for event streaming on our multi cloud architecture, which powers payments for Financial Institutions at high volumes.
This talk shares our knowledge and usage of this cool technology with the tech community.
Whistle stop tour
This talk has multiple sections, building from NATS fundamentals, continuing on to JetStream and then on to how we use NATS in production at Form3. Here are some key highlights of the talk:
- Form3’s platform is critical payments infrastructure, processing millions of payments every day across the UK and Europe.
Our old architecture relied on message fanout using SNS & SQS to send payments to our validation and gateway services, which do all the securing and mapping of the payments against the external payments infrastructure. - While this architecture was easy to start with, scale and secure, we measured a few spikes in latency of 300ms+ when our platform is processing high volumes. It was time to evolve our architecture and NATS was identified as a good solution.
- Core NATS is simple, fast and at its core is a fire-and-forget, at-most-once messaging system. You can see our demo implementations of a NATS Core Publisher and NATS Core Subscriber.
- JetStream is the recommended option for persistence and message guarantees. It has many features which were required of mature production systems, such as at-least-once delivery, data at rest encryption, horizontal scalability. You can see our demo implementations of a JetStream Publisher, JetStream Push Consumer and JetStream Pull Consumer.
- We’ve been using NATS Streaming as an event bus in our data centers in production and successfully delivering over a million messages every day.We are now going to introduce JetStream elsewhere in our payments platform. The usecase of the SNS & SQS fanout fits the NATS usage very well, so we can use JetStream as our event bus. The NATS streaming and JetStream clusters are bridged using NATS leaf nodes.
Written by
Adelina is a polyglot engineer and developer relations professional, with a decade of technical experience at multiple startups in London. She started her career as a Java backend engineer, converted later to Go, and then transitioned to a full-time developer relations role. She has published multiple online courses about Go on the LinkedIn Learning platform, helping thousands of developers up-skill with Go. She has a passion for public speaking, having presented on cloud architectures at major European conferences. Adelina holds an MSc. Mathematical Modelling and Computing degree.
Read more
Blogs · 10 min
Keeping Customers Informed During Incidents with Prometheus
Maintaining customer satisfaction during incidents is crucial for any business. In this blogpost, Piotr shares how we leverage Prometheus to expose business metrics in a secure and cost-effective way to keep customers informed and happy during those stressful situations.
May 24, 2023
Blogs · 4 min
.tech Podcast - The core components of container systems
Michael Kerrisk is a Linux expert and trainer. He joins us to explain what containers are and deep dive into the four core components of containers: namespaces, capabilities, cgroups and seccomp. He also draws parallels on how they are used by Docker to power container systems as we know them today.
May 17, 2023
Blogs · 5 min
How to inspect & manipulate network packets in AWS with Gateway Load Balancer
In this post, Michał walks you through a sample setup of the AWS Gateway Load Balancer. We will provision the infrastructure using Terraform, write a simple virtual appliance application and show it all in action. He demonstrates how this service can be used to route network traffic through a virtual appliance where each network packet can be inspected, modified, or dropped.
May 11, 2023