Acoustic Migrates 6,300 servers from 12 data centers to AWS in Just 10 Months

Executive Summary

Acoustic is a marketing technology company that provides a suite of solutions to help companies reach their customers more effectively. The firm offers a wide array of solutions, from marketing automation to mobile marketing capabilities and everything in between. Today, Acoustic serves more than 3,000 customers all over the world, including Fortune 500 companies.

Recently, Acoustic was carved out of a large multinational technology company. Acoustic needed to expand its enterprise grade product suite to the mid-market and become the leader in marketing technology. To do this, Acoustic would need to transform its technology stack to increase its rate of innovation and reduce operating costs. Acoustic was looking to partner with a cloud expert with extensive migration and modernization experience. They found that partner in ClearScale.

The Challenge

To meet its ambitious goals, Acoustic needed to upgrade its IT infrastructure on several fronts. The company wanted to have a more agile development and deployment process. Acoustic also wanted to have a more highly available solution than what was possible with its on-premises data centers. Additionally, the company was spending significantly more than desired on inflexible licenses and enterprise software. Acoustic also envisioned adding capabilities that would enable its platform to offer trial subscriptions and take advantage of pay-per-use billing. Acoustic realized that it needed to remove its dependencies on legacy hosting and database management vendors to meet all these objectives.

Eliminating its legacy dependencies would also make it easier for Acoustic to achieve modernization opportunities. As systems evolve over time, they can become more tightly coupled as new features are built on top of existing infrastructure. This can result in a very large, tightly integrated system that is brittle and difficult to maintain or modify. To break the tight coupling of dependencies, the existing application architecture would need to undergo a significant redesign. This can be a complicated and time-consuming process with inherent risk to stability and business continuity.

Achieving these goals would be challenging. Acoustic’s legacy infrastructure consisted of:

• More than 6,300 servers – both virtual and physical
• Large, complex data management workloads
• 8 full-stack applications

Moving these assets to the cloud required migrating from 12 data centers. And it would have to be accomplished on an ambitious timeline. Acoustic didn’t want to renew the contract with its large hosting provider -- a multi-million dollar fee. Avoiding that expense meant Acoustic had to migrate from its existing hosting provider in less than 10 months. Such an endeavor would typically take at least two years to complete.

Acoustic’s product suite consisted of both newly developed applications designed with microservices and legacy applications built over time. Technical knowledge of these applications ranged from active/well documented, to tribal knowledge, to no coverage. Project management and architecture design needed to be a fluid, yet concerted effort given the amount of discovery work paired with an inflexible migration milestone.

Given the scope of the project, Acoustic wanted to work closely with an experienced cloud migration expert. However, ClearScale collaborated with Acoustic leadership to propose a more comprehensive migration and modernization project to Amazon Web Services (AWS).

Since 2011, ClearScale has helped hundreds of organizations migrate, modernize, and optimize IT infrastructure for AWS, making it the ideal partner for such a large undertaking. With the team in place, it was time to move forward.

The Solution

This was a massive project that required a joint effort between ClearScale and Acoustic. It began with an in-depth analysis of Acoustic’s data, workloads, and applications. In addition to technology, the project also involved assessments on IT processes, people and skills, security and compliance practice, and cloud operations. To achieve Acoustic’s multiple business goals, the team constructed a phased approach.

Phase 1: Migrate to The Cloud

The first phase consisted of a migration from the existing infrastructure to AWS, with a subset of refactoring, to meet specific cost optimization targets. Phase one required careful planning, team coordination, and detailed research into the customer’s existing application infrastructure, deployment processes, monitoring and alerting frameworks, and data persistence tier. The team then presented a migration strategy with preliminary architecture and cost estimates to Acoustic’s leadership team. The roadmap aligned closely with Acoustic’s goals and vision, giving the company complete confidence for the migration journey under such a condensed timeline.

The teams utilized AWS’s Migration Acceleration Program (MAP), a three-step approach for migrating assets successfully to the cloud. In the first step, organizations go through a Migration Readiness Assessment (MRA) to determine their preparedness for adopting cloud capabilities. In the second step, Migration Readiness & Planning (MRP), teams establish the foundation needed to migrate successfully and build a compelling business case explaining why it makes sense to proceed. In the third step, organizations execute their migration plan, which is where ClearScale’s extensive migration background proved especially important for Acoustic.

Phase 2: Modernize Application

The second phase of the project consisted of a comprehensive modernization plan. Drawing on ClearScale’s experience in modernization of tightly coupled applications, the team was able to build out a comprehensive plan to achieve this goal. This plan involved rearchitecting the platform by decomposing large applications into many smaller microservices that are independently deployable, scalable, and follow their own lifecycles. Refactoring and modernizing an application of this scale takes careful planning, design, and implementation. ClearScale’s Solutions Architects worked closely with Acoustic’s engineers and architecture team to dive deep into each of Acoustic’s business processes.

New architecture diagrams and documentation were created for each component of the solution. Business processes were carefully analyzed and logical divisions between processes were established. For each business process, a new microservice or services were constructed to implement existing workflows and business requirements. Legacy code and business logic were reused wherever possible to reduce the time to market of the new solution. In some cases, process optimizations were identified and implemented, further increasing performance and reducing operational costs of the solution. Detailed implementation and migration plans were developed and carefully executed to ensure continuity of business and successfully migrate to the new system.

Another important shift in the modernization effort was to establish a central application message bus and migrate existing, batch-based processes to quicker, more efficient executions. This provided Acoustic an increase in overall performance while reducing the cost of the implementation. It also allowed for higher resiliency by improving processing and allowing smaller units of work to be executed and retried upon failures.

The comprehensive modernization solution also included the design and implementation of a custom data lake. The data lake contained information from streaming sources, customer provided data files, databases, and other sources. Leveraging the power of AWS resources such as AWS Glue, AWS EMR, Amazon Athena and Amazon Redshift, the team constructed a powerful data lake. This data lake allowed Acoustic to not only reduce its existing dependencies on Oracle, but also provide new opportunities for analyzing and optimizing existing workflows and finding new revenue streams for existing data.

Collaborative Transformation

Transformation is not achieved by technology alone. People need to expand their skills and adapt to changes in culture. Organizational structure and process also need to be in alignment. With an aggressive deadline for the migration, the team had to prioritize each of these efforts, delivering each just in time to create the framework from which Acoustic can continue to evolve.

The companies aligned resources at strategic and tactical levels to ensure effective planning and the ability to adapt to changes quickly. The leadership group was responsible for technology decisions and standards affecting the entire program. Program level meetings and communications were facilitated to keep the numerous execution teams in alignment and to share experiences along the way.

Given high complexity and aggressive timelines, each execution team needed to work as one cohesive unit to leverage the strengths of every member. Each team was unique and customized to the application being migrated/modernized. Acoustic resources were placed where their tribal knowledge was essential. ClearScale resources were placed where application modernization and cloud expertise made the most impact. Collaborative execution and team composition was the secret sauce that enabled transformation at this scale and velocity.

As requirements were being gathered, Acoustic was executing a mass training plan to provide their technical staff with a basic understanding of the AWS platform and services. This set the stage for a deeper collaboration as ClearScale guided Acoustic through the design process. As a result, infrastructure and application architects were able to process the new design paradigms through highly focused design sessions.

The companies agreed on a joint execution plan that involved focusing resources in areas that produced the most efficient results. Modernization efforts were prioritized based on level of risk to the migration timeline. Risks were analyzed based on application complexity, maturity of the architecture, and level of assumptions/unknowns. This was a fluid process and leadership alignment with execution teams was crucial for delivering the most value within the given constraints. Application Discovery/Design was ongoing and iterative, so priorities were updated and reviewed on a regular basis.

To ensure a timely exit from Acoustic’s large hosting provider, high risk components were migrated under the lift-and-shift pattern. These components would retain their existing configuration management (puppet/chef) and CI/CD pipelines until the Phase 2 modernization effort. Acoustic resources focused on deployment and configuration of these components with ClearScale supporting infrastructure codification using Terraform. Examples of this are several tightly coupled business processes backed by a complex legacy Oracle database.

A group of components emerged that had a high value for rebuilding completely, due to issues with scalability or application statefulness, and a low level of unknowns. This was the medium risk group. There is a high level of development effort for rebuilding, but the benefits combined with detailed requirements made modernization of this group achievable for Phase 1 migration.

The components assessed as low risk generally fell into re-platforming or containerization categories for the Phase 1 migration. This level of modernization had minimal impact on the application architecture. Acoustic was able to move Kafka, ElasticSearch, MongoDB, and Cloudant to AWS managed platforms. Acoustic developers updated application configurations to use the new services, while ClearScale managed the persistent data store migrations.

There were also several new services developed as part of the Phase 1 migration. ClearScale designed and implemented a subscription service, a customer SSO service integrated with Okta, and the billing metrics generator service. Acoustic developers worked on building a completely new UI that consolidated the products into one easy to navigate interface. These were core services shared by all products and applications. This was the foundation for unifying the products into a single platform. The new services were designed for modern cloud native architecture and run in an active-active configuration across multiple regions.

In parallel to the technical work, operational readiness efforts were also undertaken. After careful evaluation of operational processes, the teams identified two areas that needed to be revised. ClearScale guided key Acoustic resources to redefine release management and incident management and implement the processes across all product teams. This, along with a standardized CI/CD pipeline library and security and compliance processes, provided just the right level of controls and governance for the forward-thinking Acoustic Leadership team.

Just before the products started going live in AWS, the team created an operational support team to manage production requests and resolve incidents. Once the migration milestone of exiting its large hosting provider’s data center was achieved on schedule, the Acoustic operations teams joined forces with the ClearScale operational support team. The ClearScale engineers provided knowledge transfer sessions and walked the Acoustic engineers through key operational tasks. Acoustic quickly ramped up and handled all requests for Dev and QA environments with the guidance of ClearScale. The company then handed over production support and stayed on for escalations only. This approach allowed for an expedient transition through a hands-on approach.

By the time Phase 2 modernization began, the Acoustic team had built up their AWS skills and deepened their knowledge of cloud architecture. Acoustic and ClearScale were able to design the new architecture for their flagship product and mobilize five development squads to tackle the application rebuild and cut Acoustic ties to Oracle. Again, the resource mix assigned to each squad was carefully thought out and created with efficiency in mind. Three squads were led by ClearScale and two squads led by Acoustic.

The Benefits

The project was a tremendous success, thanks to the collaboration between Acoustic and ClearScale. Acoustic achieved the significant savings it desired by avoiding multiple expensive renewal fees from its legacy hosting provider. And, in addition to a more cost-efficient infrastructure, Acoustic is building new revenue by creating a pay per use SaaS model.

Acoustic’s new cloud infrastructure is also much more agile. The company’s development team can deploy infrastructure on demand and leverage a new common CI/CD library to improve the time to market for new products and features. Additionally, analysts can pull cost reporting data for each marketing product to study performance trends and identify new market opportunities.

Phase 1 Migration to the Cloud - Benefits and business outcomes include:

• Reduced infrastructure costs by moving away from existing hosting solutions
• Improved reliability with multi Availability Zone (AZ) deployments and the power of AWS
• Increased deployment speed with new deployment pipelines and tooling
• Improved maintainability through containerization and Amazon EKS
• Reduced monitoring and logging costs through use of AWS CloudWatch
• Moved high value services out of a single Relational database into individually scalable persistence stores, further optimizing costs
• Laid framework for modernization effort in phase 2

Phase 2 Application Modernization - Benefits and business outcomes include:

• Increased scalability to optimize infrastructure costs and react in real-time to fluctuating customer workloads
• Improved application reliability by automatically replacing instances that experience issues without impacting availability
• Migrated services to stateless services, reducing probability of data loss in the event of a hardware or software failure
• Improved data durability by leveraging the reliability of AWS services such as S3 and DynamoDB
• Increased speed of feature delivery and patches to production
• Trained Acoustic team members in cloud computing best practices and acted as a force multiplier for the customer
• Decreased the maintenance burden of a legacy infrastructure
• Improved performance of long-running customer initiated processes
• Hardened systems against failures and lessened probability of outages by reducing blast radius of any particular issues
• Constructed a custom data lake to facilitate in depth analytics and business intelligence

Summary

Through this partnership, Acoustic was able to ignite business transformation by changing their culture, processes, and technology. Acoustic is now well-positioned to win new mid-market customers, as its IT infrastructure can now support the “try-before-you-buy” sales model. Acoustic can provision trial accounts on-demand for potential customers using infrastructure-as-code principles. On top of that, satisfied marketers can easily convert to paid subscriptions for most products.

Finally, by moving from on-premises data centers to the cloud, Acoustic’s portfolio of marketing SaaS offerings benefit from AWS’s high availability services, providing customers with a performant and reliable service.

With these capabilities, Acoustic is positioned for long-term success as the marketing technology leader in today’s fast-moving global marketplace. The company was able to avoid a massive renewal fee, break ties from legacy IT dependencies, migrate to a leading cloud platform, and modernize its infrastructure to leverage advanced cloud technologies.

The success of this transformation is a product of the partnership established between Acoustic and ClearScale.