.tech Podcast - Pulumi

David Flanagan from Pulumi gives us an introduction to Infrastructure as Code (IaC) and Pulumi. Then, he explains how Pulumi executes plans, model dependencies and orchestrates systems. Finally, he discusses the differences between Pulumi and Terraform.

David is a Staff Developer Advocate at Pulumi. He spends a lot of time creating developer content such as codified examples, tutorials and conference talks to help technology users. He also produces a lot of cloud native educational resources on his excellent YouTube channel - Rawkode Academy.

Infrastructure as code (IaC)

As a cloud customer, you have access to a management interface which allows you to provision VMs, bare metal resources, load balancers etc. This is a non-repeatable, error prone process.

Infrastructure as code (IaC) is the inverse of that, where we don’t rely on manual configuration in the UI. Instead, IaC allows us to programatically describe and create those resources by communicating with the cloud provider APIs.

This provides us a with:

• A repeatable process, which lifts the burden of environment configuration from developers and operation teams.
• A deterministic environment for all use cases, which avoids variance and bugs in environment configurations.
• The ability to check in infrastructure configuration into source control, making it easy to review and see how it changed.

What is Pulumi?

Pulumi is a command-line application that provides software development kits (SDKs) for cloud providers and more. It gives you a data structure model that you can use to describe resources.

It allows you to write code and create classes in a variety of languages - Go, .Net, Typescript, Javascript, Python. When running, Pulumi will create a directed acyclic graph (DAG), which will work out dependencies and create the resources in the correct order. Furthermore, Pulumi runs are idempotent and reconcile drift (undeclared changes which).

Pulumi is different in that it allows you to use the programming languages you are already comfortable with rather than a domain specific language(DSL).

Executing Pulumi plans

The imperatively declared Pulumi plans output declarative resources, which is something that be hard to get your head around. You can write declarative code using imperative code, which is what functional programmers do.

Under the hood, Pulumi executes the code, monitors the resources that have been created, stores them and reconcile what it needs to do afterwards with the cloud provider APIs.

Every resource created by Pulumi gets a unique identifier. It breaks everything down to CRUD operations, using the read operation to detect if any updates need to be made. For example, if you were to change your resource count from 2 to 5, the Pulumi would detect that change and spin up 3 new instances to bring the total count to 5. Pulumi is also able to detect which updates can be made in place and which updates require a replace.

Managing resource changes is a complex process, so using and IaC tool will help your developer teams handle this.

Modelling dependencies in Pulumi

Dependencies inform the order of creation and destruction of resources.

Resources will be created from top to bottom in the code that you have written, unless dependencies are manually declared. This will allow you to control the creation and destruction of your resources.

Pulumi has an input-output type system. It will determine dependencies based on this type system, but you can also manually change it. For example, you want to create a load balancer, but will need to pass it an IP address from a VM or similar. Pulumi will know to spin up the VM first, then pass that IP address to the load balancer when it is ready.

A common pitfall is to hardcode resource values, even though you might be able to work it out. The IaC tool will not know there is a dependency between those resources if you use hardcoded strings. The type systems in the Pulumi supported languages help enforce that as well.

Orchestrating projects with Pulumi

Most organisations now have a platform engineering team that are in charge of managing the increasingly complex infrastructures that we run.

The typical setup in Pulumi is:

• a Git repository that defines your core infrastructure, typically a managed Kubernetes service
• CI/CD integration for most popular technologies, which creates base resources and provides feedback
• a stack reference which allows your application teams to subscribe to your stack

Pulumi also supports GitOps, which is the practice of platform engineering teams installing Kubernetes operators in the cluster and pulling in the correct repositories. This is a good developer experience, as they don’t have to worry about stack references. They just push their code to their repositories and the operator makes it happen for them.

Pulumi provides their own Kubernetes operator, which is a great starting point when setting up new infrastructures. From there, the deployment pipeline is controlled via Git.

Differences from Terraform

People often think of Terraform as synonymous with IaC. Terraform has a DSL called Hashicorp configuration language (HCL). It is used to describe all Terraform programs and it does have some constraints, even though it has been adding new features lately.

One of the main differentiators of Pulumi was its choice to not use a DSL at all, but to use the imperative languages that developers are already used to, with a wide array of supported SDKs.

These programming languages come with a host of benefits:

• Solved distribution such as npm, PyPy etc.
• Support developer favourite tooling
• Support testing your code. Pulumi provides mocking and assertion frameworks to enable you to unit test your code.

Most of all, Pulumi wants to remove the cognitive overload of HCL, making it easier for developers to get involved in infrastructure.

Interested in being a guest speaker?

If you enjoyed this episode and would like to be part of the podcast, then please fill in this form and we’ll be in touch. ✍️