How to Bring the Magic of Automation to Mulesoft Anypoint Platform
If you are reading this article, I assume you know what Mulesoft Anypoint Platform is. For those who don’t know, Anypoint Platform is a complete solution for API-led connectivity.
The focus of this article is on the automated provisioning of the below Integration Platform as a Service(Cloudhub) components.
Manual/Repetitive Application Management Tasks
Deployment of Mulesoft APIs in Cloudhub involves creating and updating resources in the Anypoint Platform components like API Manager, Runtime Manager and MQ.
API Manager
- Creation and Management of the APIs in the API Manager from the API specifications published in Anypoint Exchange
- Making a note of the API Id of the API instance which is used for Autodiscovery Scheme in the Mule application’s apikit configuration
- Applying the API level policies (if any) to the API instance in API Manager
Runtime Manager
- Deploying the mule application in Runtime Manager with your deployment pipeline
- Applying the application settings (if any) in the Runtime Manager like
Disable Cloudhub logs
,Logging level and packages
,Static Ip (enable and allocate if required)
andAuto restart when application is not responding
MQ
- Provisioning of Queues and Exchanges in Anypoint MQ referenced by the deployed applications
- Assigning Dead Letter Queues
- Binding Queues to Exchange
Access Management and Secrets
- Managing/Storing Anypoint environment credentials (clientId and client secrets) and the environment-specific encryption key used by applications for handling secure properties
One can argue that most of the above tasks are one-timers as the API instances, queues and exchanges are not updated often concerning the frequent application deployments. It could be right for a few teams, but I think they are missing the bigger picture.
Firstly, even if we perform the above tasks only once, still it’s done manually over the Anypoint Platform portal which should be automated with CI/CD pipelines.
Secondly, it becomes quite cumbersome when we have to spin up a new Cloudhub environment and deploy all the Mulesoft APIs with environment-specific configuration.
“Wouldn’t it be awesome if we can spin up an entirely new Cloudhub environment and provision all the required resources by just reading a declarative environment configuration file (YAML or JSON )” 🤓
“How can we adopt Infrastructure as Code(IaC) approach while provisioning a Cloudhub environment?” 🤔
Environment Provisioning using Infrastructure as Code (IaC)
Infrastructure as Code(IaC) automates the provisioning of infrastructure, enabling your organization to develop, deploy, and scale applications with greater speed, less risk, and reduced cost. It uses high-level environment configuration files to automate the provisioning of the infrastructure.
As of now for provisioning Cloudhub environments, there’s no predefined Infrastructure as Code template. I came up with a sample YAML file which defines the environment configuration. It includes the provisioning details required for API Manager, Runtime Manager and Anypoint MQ. The structure of this file will be different for every organization, based on the resources to be provisioned.
Here’s the JSON equivalent of the above environment configuration YAML file.
“Creating an environment configuration file was easy” 😊
“Now I have a huge task ahead of me to parse it and provision the environment. Do I need to build a custom software tool to do this job? Something like Terraform” 🤷♂️ 💭
Anypoint Platform APIs to the Rescue
The Anypoint Platform APIs enable you to programmatically access the functions of API Manager, Runtime Manager, MQ and whatever actions you could perform in the Anypoint Platform UI portal. Moreover, every time you log in to the Anypoint Platform, the UI implicitly invokes the RESTful Platform APIs.
For more straightforward automation tasks, Anypoint CLI, which is a wrapper of Platform APIs, is enough. But by leveraging the power of Anypoint Platform APIs, you could build a robust software tool which parses the environment configuration file and provisions the defined Anypoint Platform resources.
It might sound daunting to build such an API wrapper. Still, honestly, if you know the right Platform APIs, it gives you immense flexibility to automate the build or tear down of environments.
Building Anypoint Platform API wrapper: Where to Start
The first step towards achieving this goal is to define the structure of the environment configuration file. This file should be easily extendable for future environments to be provisioned. It might sound like the easiest thing to do, but if not done correctly can make you pay the price.
“Design and define your environment configuration file diligently”
Next step is to implement the IaC approach to automate the provisioning of the defined infrastructure and deploy the applications/API instances/MQ. It requires building a parser which parses the configuration file and performs the provisioning of the defined Anypoint Platform components. To begin with, you can adopt an imperative or procedural approach which lets you perform one specific step at a time in the appropriate order to end with the desired state of the environment.
By harnessing the power of Platform APIs, you can create a custom command-line interface(CLI) which can be leveraged in the CI/CD pipelines, DevOps scripts or your local machine. Below are the screenshots of a custom CLI implemented by me.
And if a custom CLI is not enough for you, how about building a slack command to trigger the automation or an Alexa skill to know the status of the deployed applications.
/anypoint build-env dev
/anypoint teardown-env dev
There are no limits to what you can build as a wrapper to the Anypoint Platform APIs.
Taking Automation to the Next Level
The imperative approach to automation using IaC gives you more control while executing complex commands. Still, things go south when one of the commands fail while building up a new environment or tearing it down. The environment is left in an unknown state. As a result, the DevOps team is required to execute a few rollback commands to get the environment in a correct state.
What if we take our automation to the next level by adopting the Declarative approach using IaC. It will enable us to preserve the snapshot of the environment before we perform its build or teardown. This facilitates automated rollbacks without any manual intervention. Additionally, it creates an execution plan only for the resources which are either new or updated, as in Terraform or Cloudformation.
Once you reach this level, you may plan to launch a new software tool to provision Anypoint Platform components and name it “CloudhubFormation.” 😄
References below will help you get started with Anypoint automation.
References
- Anypoint Platform APIs Exchange Assets
- Cloudhub API Specification
- Anypoint MQ Admin API Specification
- API Manager API Specifications
- Getting started with Mulesoft Platform APIs