Introduction to Infrastructure as Code (IaC)

What is IaC?

Infrastructure as code or IaC enables developers to programmatically create, deploy and manage cloud resources in an automated, consistent and scalable manner. Notice the emphasis on scalable – that means the IaC template will spin up the same resources with the same configuration every time unless the cloud provider itself changes its configuration. This reduces the operational overhead of creating cloud resources, enabling developers to focus on delivering high-quality software and services to their customers.

Why do we need IaC?

Before IaC, developers would use a ‘Click-Ops’ method to create resources; essentially clicking on buttons, following the prompts, and referring to documentation if they get stuck. Alternatively, some developers would opt for cloud provider’s own CLI such as AWS CLI or Google Cloud Shell to deploy resources.

Using Click-Ops or the CLI can be a quick and straightforward way to create resources on the cloud, especially for small-scale projects and quick prototyping. It can be useful for small, one-off tasks or for exploring the capabilities of the cloud provider. But what if you were leading a team of 10 data engineers and data scientists and you want everyone to use the same cloud stack? You could create a guide and tell them to follow the setup themselves, however, it can quickly become cumbersome and error-prone when managing a large number of resources or complex infrastructure.

To address these issues, IaC tools are developed.

Types of IaC tools

There are two types of IaC tools – ones that are built in-house by cloud providers, and open source.

IaC tools by Cloud Providers

• AWS CloudFormation: AWS CloudFormation is a service that allows you to define your infrastructure as code using JSON or YAML. CloudFormation supports a wide range of AWS services and resources and also allows you to create custom resources using AWS Lambda.
• Azure Resource Manager (ARM): Azure Resource Manager is a service that allows you to define your infrastructure as code using JSON or YAML. ARM supports a wide range of Azure services and resources.
• Google Cloud Deployment Manager: Google Cloud Deployment Manager is a service that allows you to define your infrastructure as code using YAML or Jinja2 templates. Deployment Manager supports a wide range of Google Cloud Platform services and resources.

Open Source

Terraform

Terraform is an open-source IaC tool that allows you to define your infrastructure as code using a declarative language called HashiCorp Configuration Language (HCL) or JSON. HCL is the recommended language as it’s explicitly designed for Terraform. It currently enjoys a dominant position among open-source IaC platforms.

To deploy an AWS S3 bucket with Terraform, you will need to follow these steps: a. Define the S3 bucket in your Terraform configuration file:

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "my_bucket" {
  bucket = "my-bucket-name"
  acl = "private"
  
  versioning {
    enabled = true
  }

  tags = {
    Environment = "dev"
  }
}

b. Initialize Terraform in your project directory by running terraform init. c. Create a Terraform execution plan by running terraform plan. This will show you the changes that Terraform will make to your infrastructure. d. Apply the Terraform execution plan by running terraform apply. This will create the S3 bucket in your AWS account.

Pulumi

Emerging as a fierce competitor to Terraform, Pulumi is a universal infrastructure as code platform that allows you to use familiar programming languages and tools to build, deploy, and manage cloud infrastructure. To deploy an AWS S3 bucket with Pulumi, you will need to follow these steps:

a. Use the pulumi_aws Python library to create a resource.

import pulumi
import pulumi_aws as aws

# Create an AWS resource (S3 Bucket)

my_bucket = aws.s3.Bucket("my-bucket",
                          bucket="my-bucket-name",
                          acl="private",
                         )

# Export the name of the bucket
pulumi.export('bucket_name',  bucket.id)

b. Running pulumi up in the terminal will create the S3 bucket in your AWS account.

Terraform vs Pulumi

Both Terraform and Pulumi support a wide range of cloud providers, including AWS, Azure, and Google Cloud. The main difference between Pulumi and Terraform is that Pulumi allows you to define your infrastructure using a general-purpose programming language, while Terraform uses its own declarative language (focuses on the what) called HashiCorp Configuration Language (HCL) or JSON.

With Pulumi, you can use popular programming languages such as Python, JavaScript, Go, and TypeScript to define your infrastructure. This allows you to leverage the full power of a programming language to define, configure, and deploy your infrastructure. Pulumi also provides a set of libraries for working with cloud providers, allowing you to easily create and manage resources.On the other hand, Terraform is designed specifically for infrastructure as code and provides a domain-specific language (HCL) that is optimized for describing infrastructure resources. Terraform also has a large ecosystem of providers, which allows you to manage a wide range of cloud resources.

Here are some additional differences between Pulumi and Terraform:

• Pulumi has a more procedural approach (how), while Terraform is more declarative (what).
• Pulumi supports more cloud providers than Terraform, including AWS, Azure, Google Cloud Platform, and Kubernetes.
• Pulumi allows for easier refactoring and reuse of infrastructure code, as it uses a programming language that is familiar to developers.
• Terraform has a larger community and ecosystem of providers, making it easier to find resources and examples for managing specific cloud resources.

Ultimately, the choice between Pulumi and Terraform depends on your specific needs and preferences. If you prefer a general-purpose programming language and want more flexibility in defining your infrastructure, Pulumi may be a good choice. If you prefer a declarative approach and want to leverage a deeper and more stable knowledge base, Terraform may be a better fit.