Optimizing CDN Management Using Terraform

What is Terraform?

‍

Before we dive into what Terraform is, let's begin with a real-world example of how Terraform code looks and works:

‍

# Challenge requests coming from known Tor exit nodes.

‍

	
  resource "cloudflare_access_rule" "tor_exit_nodes" {
  zone_id = "0da42c8d2132a9ddaf714f9e7c920711"
  notes   = "Requests coming from known Tor exit nodes"
  mode    = "challenge"
  configuration {
    target = "country"
    value  = "T1"
  }
}
  

‍

In the example above, we're using Terraform to set up an access rule on Cloudflare. This rule challenges requests coming from known Tor exit nodes. The code is declarative, which means you specify what you want the final configuration to look like, and Terraform ensures the infrastructure matches that desired state.

‍

So, What Exactly is Terraform?

‍

Terraform is a revolution in the way we handle infrastructure. It enables DevOps teams to manage, configure, and automate infrastructure in a streamlined fashion. It’s not just limited to cloud resources like AWS and Azure; Terraform is versatile, extending its capabilities to key performance areas like Content Delivery Network (CDN) management, ensuring efficient content delivery and optimal user experience.

‍

Started years ago with a focus on cloud management, Terraform built a massive community around its open-source license–up until two weeks ago, but more on that later. The community even developed a boatload of plugins to manage various tools and services; this includes comprehensive management of CDNs such as Cloudflare, Fastly, Akamai, and many others, allowing businesses to integrate and automate their content delivery mechanisms seamlessly.

‍

{{promo}}

‍

Background

‍

The idea of managing infrastructure through code wasn't initiated by HashiCorp. In fact, Amazon Web Services (AWS) paved the way with its service, CloudFormation, which introduced the concept of Infrastructure as Code (IaC). CloudFormation allowed users to utilize code to define and provision AWS infrastructure resources in an automated and consistent manner.

‍

However, where AWS provided the foundation, HashiCorp, the company behind Terraform, built an empire. Taking inspiration from AWS's CloudFormation, HashiCorp introduced Terraform with a broader vision. 

‍

Unlike CloudFormation, which was confined to AWS resources, Terraform was designed with a multi-cloud approach in mind. This means that with Terraform, you can manage resources across multiple cloud providers, including AWS, Azure, Google Cloud, and more, using a single tool.

‍

A significant factor that contributed to Terraform's rise to prominence is its open-source nature. Being open-source, Terraform invited a vast community of developers and companies to rally around it. This led to the creation of numerous plugins, enabling Terraform to interact with a myriad of tools and services beyond just cloud providers. A prime example is the Cloudflare Terraform plugin, which allows users to manage Cloudflare configurations using Terraform.

‍

Companies, big and small, leveraged Terraform's open-source flexibility to develop plugins tailored to their specific needs, thus extending Terraform's capabilities even further. Take Cloudflare as an example; they have an entire service called Cloudflare Terraform. 

‍

Today, due to its adaptability, versatility, and strong community support, Terraform stands as the industry standard for infrastructure management through code.

‍

‍

Why Manage Infrastructure by Code?

‍

‍

If you’re still grappling with bothersome UI interfaces or shuffling through multiple dashboards to manage your infrastructure, it’s high time you join the IaC revolution. 

‍

Version Control

First, let’s talk about version control, an essential aspect you stand to gain by adopting IaC. You can integrate infrastructure management into Git repositories, making it easier to track who did that, when, and why. This offers not just an audit train but an “audit trail on steroids.”

‍

A simple ‘git blame’ can point you right to the individual who made a particular change.

‍

‍

‍

Real-Time Monitoring & Anomaly Detection

In the IaC paradigm, code changes often trigger automatic re-deployments. Real-time monitoring can help detect any anomalies as soon as they happen, thereby increasing the speed at which problems can be diagnosed and solved. 

‍

It takes away the manual process of checking logs and monitoring dashboards, giving your team more time to focus on creating value. 

‍

Accountability and Transparency

Then comes the ‘whodunnit’ factor. With code, you can easily identify who made a specific change, unlike UI, where these changes often get buried in logs that no one really checks until something goes wrong. 

‍

Through platforms like GitHub, you can track changes down to the specific line of code and the person behind it. This level of granularity is not found in UI-managed systems. 

‍

Automation

Code-based systems also shine in automation. When you codify infrastructure, you can integrate it with DevOps pipelines to automate various tasks like server provisioning, database scale, and security audits. If attempted through a UI, this same automation will be much harder and error-prone. 

‍

Collaboration

Infrastructure as Code (IaC) isn't just a tool for automation—it's a catalyst for collaboration. With the entire infrastructure codified, specialized teams, including DevOps and DevSecOps, can operate with enhanced synergy. This dynamic is particularly transformative for organizational behavior.

In traditional setups, teams like Security and Operations often functioned in silos.

‍

With IaC, these traditionally distinct teams find themselves compelled to integrate more closely. One of the primary reasons is the shared workflow. Infrastructure code, like any software code, can be reviewed, discussed, and modified collaboratively.

‍

A crucial aspect of this collaboration is the workflow. For example, a DevOps team can set up a workflow that scrutinizes the code pushed by SecOps. This workflow can be configured to automatically block or allow changes based on predefined criteria, ensuring that security configurations align with operational requirements. 

‍

This workflow not only streamlines processes but also ensures that any changes are in line with the organization's best practices and compliance standards.

‍

Advantages of Using Terraform

‍

From its clown-agnostic approach to its modular architecture, Terraform not only serves as the backbone of your infrastructure but does so in a way that respects the complexities of modern cloud infrastructure at scale. 

‍

1. Scalability

One of Terraform's standout features is its inherent ability to scale resources effortlessly. As businesses grow and demand fluctuates, infrastructure needs to adapt. Terraform's declarative nature makes this adaptation straightforward.

‍

Consider an example where a company is running multiple virtual machines (VMs) on a cloud provider. If the initial requirement was for 5 VM instances and the demand suddenly increased, adjusting the infrastructure to meet this demand is as simple as modifying a single variable in the Terraform manifest.

‍

Terraform Code Example:

variable "instance_count" {
  description = "Number of VM instances"
  default     = 5
}


resource "aws_instance" "example" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
  count         = var.instance_count
}

‍

In the code snippet above, the number of VM instances is controlled by the instance_count variable. To scale from 5 to 10 machines, you simply adjust the default value of this variable from 5 to 10 and rerun the Terraform apply command.

‍

Beyond just VM instances, this scalability extends to other aspects of your infrastructure. Whether you're scaling storage solutions like S3 buckets, compute resources like EKS clusters, or content delivery mechanisms via CDNs, Terraform offers a streamlined approach. 

‍

Businesses can adjust configurations to manage everything from basic distribution settings to intricate geo-targeted content caching and delivery strategies, all with the same ease.

‍

‍

2. Easy Maintenance

When we say “easy maintenance,” it’s not a vague promise. With Terraform, you can compartmentalise your code into modules, allowing for a cleaner, more understandable codebase. 

‍

Say, your front-end and back-end resources are segmented into two different modules. If an upgrade is needed on the front end, simply modify the corresponding module and execute Terraform’s ‘apply’ command, leaving the back end untouched but still under version control. 

‍

3. Enhanced Control & Readability 

With a typical GUI, users often find themselves navigating through a maze of checkboxes and form fields. While these graphical interfaces can be intuitive, they often lack a comprehensive view of the entire configuration. How confident can one be about the exact settings applied through a series of clicks?

‍

Terraform, on the other hand, presents a stark contrast. Its configuration files meticulously detail every attribute of your resources. These files serve not just as configuration but as a manifest—a clear, concise blueprint of your infrastructure. It's a declaration that encapsulates your entire setup.

‍

The strength of Terraform's approach shines in its readability. With Terraform, users have the ease of overviewing their entire configurations in one place. 

‍

There's no need to go through multiple tabs or screens. Every setting, and every resource attribute is laid out in a structured format. This ensures that anyone, from seasoned developers to newcomers, can read, review, and understand the infrastructure at a glance.

‍

‍

4. Redundancy

Redundancy is all about having an automated, replicable backup. The Terraform scripts you write not only serve as documentation but also as a disaster recovery plan. 

‍

If a part of your infrastructure was to fail, you could simply refer back to your version-controlled configurations and restore the lost resources in a matter of minutes. 

‍

For CDNs, redundancy is really important. In the event of a specific Point of Presence (PoP) failing, businesses need an immediate fallback. With Terraform's scripted infrastructure, businesses can ensure their content is always available, swiftly rerouting traffic when necessary.

‍

Additionally, as part of a Continuous Deployment (CD) pipeline, there is always a code review that sees the exact changes, helping to track and mitigate malfunctions.

‍

‍

Disadvantages and Limitations

‍

Terraform is a big deal in the infrastructure-as-code (IaC) world, but like any tool, it has its shortcomings. 

‍

Without sugarcoating anything, here’s what you need to know:

‍

Provider Dependency

Terraform relies on providers, which are essentially plugins that allow you to manage specific services, such as AWS, Azure, or even non-cloud stuff like GitHub. 

‍

But here's the catch: if the provider lacks certain features, you're handcuffed. Imagine building a high-tech car but only having access to a basic engine. It might get the job done, but you're missing out on potential horsepower.

‍

In this case, all the Terraform features are not API-ready, meaning you’d be stuck with a limited set of tools if you choose to harness this tech.

‍

‍

‍

Complexity and Learning Curve

While Terraform can be remarkably flexible and powerful, it demands you to learn its language, HCL (HashiCorp Configuration Language). 

‍

If you're coming from a world of JSON or YAML, this might look alien initially. The syntax isn't the only barrier; the concepts themselves can be complicated for newcomers.

‍

‍

Migratory Nightmares

You might think that if you've set up your infrastructure in AWS using Terraform, shifting to Azure would be a walk in the park. Hold that thought. Different providers have different plugins and configurations. Migrating isn't as simple as 'copy-paste'.

‍

Even within the same cloud provider, let's say, moving from an EC2 instance to a Lambda function in AWS, could force you to rewrite your entire codebase, or migrating from one CDN/Cloud to another with Terraform can take up to a year. 

‍

Transitioning between CDN providers or even changing CDN configurations can be tricky. While Terraform provides a level of abstraction, the inherent differences in how CDNs handle caching, invalidations, and routing can require significant reconfiguration.

‍

‍

Terraform is a Standard

‍

‍

The prominence of Terraform in the infrastructure world isn't just a stroke of luck. We've reached an age where efficient and swift infrastructure deployment is more crucial than ever. In this fast-paced era, the ability to manage infrastructure programmatically, reliably, and consistently is indispensable.

‍

The dawn of Terraform marked the beginning of a new era. Its approach to Infrastructure as Code (IaC) streamlined infrastructure management in ways that weren't previously feasible. Think about it: Infrastructure, a once predominantly manual and time-consuming affair, can now be handled as effortlessly as writing and deploying code.

‍

But how does this apply to CDNs? A Content Delivery Network’s job is to distribute content in a fast, efficient, and reliable manner across the globe. To achieve this, there's an underlying layer of infrastructure that needs constant configuration, monitoring, and scaling. This is where Terraform shines.

‍

‍

Optimized Infrastructure

Integration and API Support: One testament to Terraform's authority in the realm is its universal API support. It’s hard-pressed to find a reputable CDN or any infrastructure tool that doesn't have Terraform integration. 

‍

The logic here is clear: for a tool to be respected and remain competitive in the infrastructure space, it must play nicely with Terraform. The tool's extensive provider support ensures that integration is smooth and versatile, allowing us to communicate with various CDNs via their APIs and manage them programmatically.

‍

The Pro DevOps' Choice: Now, imagine you're a DevOps professional. Your role involves managing the complex infrastructure that powers CDNs. Would you want to deal with endless UI configurations, or would you prefer writing code that deploys and scales infrastructure seamlessly? 

‍

Of course, the latter. And that's exactly why any DevOps guru you meet today will swear by Terraform. It not only gives them the power to manage infrastructure but also ensures that their CDN configurations are optimal, secure, and scalable.

‍

{{promo}}

‍

Common Practices in Terraform

‍

Picture this: We have two expert teams in a company - the DevOps team and the DevSecOps team.

‍

1. The DevOps Team: Think of these individuals as the backbone of the company's infrastructure. Their expertise lies in ensuring everything runs smoothly, from servers to networks. They're like the architects and builders of a skyscraper, ensuring it's stable, functional, and can withstand any challenges.

‍

2. The DevSecOps Team: These are the specialists focusing solely on security services - things like Web Application Firewalls (WAF), Rate Limiting (which prevents a system from being overwhelmed by too many requests), and Bot detection (to fend off malicious automated attacks). If the DevOps team builds the skyscraper, the DevSecOps team ensures it's burglar-proof and fire-resistant.

‍

Now, here's the twist: even though the DevSecOps team handles security, they don't have free rein to make any changes they want. Why? Because the entire infrastructure's responsibility lies with the DevOps team. They hold the master key. Any adjustments, especially critical ones relating to security, need to be cross-verified to ensure they don't inadvertently cause other issues.

‍

So, what happens if the DevSecOps team believes a change is needed? They can't just apply it. Instead, they draft their proposed changes and push this code to a platform called GitHub. Think of this as submitting a proposal or a plan for review.

‍

The DevOps team then reviews this 'plan' on GitHub. They check it thoroughly, ensuring it's in line with all other systems in place and won't cause any hiccups. Only with their approval can these changes be implemented.

‍

‍

What Happened Two Weeks Ago?

‍

Earlier this month, HashiCorp, made an announcement that took many by surprise. They revealed a decision to change the open-source license for Terraform.

‍

The switch was not just a minor licensing change; it was a move from the Mozilla Public License v2.0 (MLP 2.0), a renowned open-source license, to the Business Source License (BSL) v1.1. Now, for the uninitiated, BSL is a “source-available” but not traditionally regarded as open-source. 

‍

This means while the source code is available for viewing, it comes with certain restrictions that do not align with the orthodox open-source philosophy.

‍

‍

The Reaction of the Open-Source Community

No sooner did the announcement make its way to public forums; the open-source community found itself in a whirlwind of discussions, debates, and, unfortunately, discord. 

‍

Terraform, with its legacy and vast user base, had cultivated a strong community over the years, and many felt the licensing change was a betrayal of sorts. 

‍

It wasn’t just about a change in code accessibility; it was about trust. Given Terraform’s standing in the open-source arena, the announcement made many question the future of other HashiCorp products. Would they follow suit? The skepticism wasn't unfounded; after all, in the tech world, one move can set a precedent. 

‍

OpenTF - A New Fork

Within days, ripples of dissent began taking shape. By Friday, a splinter group from the original Terraform community made their move. Announcing their intent to develop an open-source fork of Terraform, they launched the OpenTF Project. 

‍

OpenTF was not a product; it was a statement. A stand for the principles of open source and a testament to the community’s resilience. It aimed to carry forward the open-source nature of Terraform under the genuine open-source umbrella of the Linux Foundation. 

‍

‍

Bottom Line

‍

Open source has been a double-edged sword for companies like HashiCorp. While it fosters community and collaboration, it often doesn’t translate into revenue, especially when large cloud providers take the code and commercialize it. 

‍

However, the response to HashiCorp’s decision reveals a stark disconnect between business strategies and community expectations. Only time will tell if Terraform’s future includes a middle ground that satisfies both. 

‍

‍

‍

‍