To provide an environment for rapid development and deployment of integration applications, we helped a Fortune 500 insurance company build a custom platform, from the ground up, that was similar to private Cloud platforms today.
Years ago, one of the largest insurance companies in the United States, located in the Midwest, was planning a complete overhaul of their software application infrastructure. They had thousands of applications that supported their business and wanted a consistent, reliable, and efficient way for their internal clients to find and consume these applications. The scale of the initiative was overwhelming and would have taken decades to fully implement with a traditional development strategy and environment. The challenge was how to build, deploy, test, and release applications in days, not weeks or months.
The primary objective was to provide a platform to the business that minimized the full application development lifecycle and allowed consumption and management of applications as quickly as possible, with as little latency as possible.
Several tools were chosen to meet the objective. The software involved was Mule Runtime, RabbitMQ, Oracle DB, Splunk, Puppet, and RPM (RedHat Package Manager).
Simplify the Environment by Minimizing the Toolset
Hundreds of developers were involved with the project. In order to quickly and efficiently on-board developers new to the environment, the toolset within Mule was simplified. Only three message processors were initially allowed to be used within the Mule applications. Not only were the applications light weight and focused on a single task, but the time to perform code reviews and testing was significantly reduced for each application.
Developing a repetitive and reliable integration environment that was used by hundreds of developers was tricky. Developing that environment in conjunction with monitoring tools to efficiently track messages throughout the organization, while maintaining a very aggressive SLA for our business groups, was very
tricky. The head of the group was the brains behind the platform. While Mule was new to him, he knew what needed to be done and found a way to do it. The team consisted of approximately ten people, though it changed from time to time. Our part was to provide the Mule expertise to help get everything done.
The first several months were spent building the infrastructure for Mule. This piece involved developing the MuleSoft framework to develop, test, and monitor the integration environment. The remainder of the time was spent developing and maintaining the self-service platform for the Mule virtual machines in the Integration layer. While both parts were vastly different, they came together to form a flexible, reliable, and efficient platform that is similar to many of the containerized platforms that have been developed over the years.
We accomplished three overall items. First, the group developed a tracking mechanism that logged events with extremely low overhead to the messaging architecture. Next we integrated it with Splunk to provide a reporting framework for users at all levels. Finally, we configure Mule to work with RabbitMQ and Oracle, using a very limited set of Mule capabilities (HTTP and AMQP). There were many smaller facets that were involved those three items, that helped the platform succeed.
The next phase of the project involved creating a self-service deployment portal for “Integration Packages”. The first task was to create a server group with Puppet that allowed us to create server cluster images with Mule Runtime and RabbitMQ on each node. This was Docker-like process that spun up a set of servers within minutes. Each set of clustered servers hosted a single Mule application that had an optional RabbitMQ node. To bring this together, RPM’s (RedHat Package Manager) were used to build the Mule applications and package them into an installation file. The final deployment process created the server images and then installed the RPM’s on the newly created servers.
The total project involved millions of man-hours with thousands of people. Our job was to assist with creating the infrastructure for the integration platform. We were on the project for just over two years and helped setup a home-grown, self-service, private Cloud infrastructure that provided a reliable and efficient platform for thousands of people.
This was a greatly satisfying and challenging project that introduced us to strategies used to build many of today’s PaaS offerings.