Pathlight Executes Large-scale Database Migration to AWS

Executive Summary

Pathlight is a Realtime Performance Management (RPM) platform that allows large, customer-facing teams to move faster than ever before. By consolidating performance data, coaching, and communication in one place, the platform empowers data-driven management at every layer of the organizational chart.

The company keeps data about its everyday operations. Over time, Pathlight outgrew its data warehouse and underlying architecture, kicking off the search for a better long-term solution. After executing a proof of concept (POC), the organization asked ClearScale for assistance with a full-scale migration.

The Challenge

Pathlight’s platform has evolved significantly over its long history. As the organization has grown, the legacy data layer has struggled to keep up, holding Pathlight back from scaling effectively. The IT team has tried scaling instances up at different times, but these updates have never served as more than temporary stop-gaps. As a result, Pathlight’s leadership team knew that the time had come to consolidate the organization’s multiple single-tenant data warehouse databases into a single data warehouse and leverage the convenience of a fully-managed database service.

Pathlight decided to move forward with a POC and create a runbook for a database migration to Amazon Aurora. The team successfully executed the POC but wasn’t convinced its process would work for a full-scale migration involving a dozen databases. Plus, IT had competing priorities, including the need to manage the existing RDS instances. They didn’t have the capacity to handle a much bigger endeavor. The IT team was swamped with backlog items, motivating Pathlight to find a cloud expert that could augment the in-house team and make helpful recommendations.

Pathlight found ClearScale, an Amazon Web Services (AWS) Premier Consulting Partner with deep database migration experience, and asked the company to validate the proposed migration strategy. Pathlight also tasked ClearScale with helping the organization move from RDS for PostgreSQL to Aurora for PostgreSQL, a fully-managed database service that offers commercial-grade performance at a much lower cost than on-premises solutions.

The ClearScale Solution

ClearScale first evaluated Pathlight’s runbook and POC migration process. ClearScale’s engineers determined that Pathlight’s process would work for small and medium-sized databases but would fall short for larger databases. Pathlight didn’t have the storage bandwidth needed from its database host to handle its current Amazon Elastic Block Storage (EBS) volume. ClearScale also didn’t think Pathlight could execute its migration in an acceptable amount of downtime without making key changes to the runbook.

ClearScale modified Pathlight’s runbook for larger instances and implemented a temporary copy of the source database to accelerate the migration timeline. For the actual migration, ClearScale engineers provisioned temporary instances (db.m5.8xlarge), imported database backups, and sent data to Amazon Aurora over Pathlight’s existing pipelines.

Along the way, ClearScale helped Pathlight consolidate its databases under a single data warehouse and set the organization up with AWS managed services. Doing so freed up Pathlight’s IT department from having to manage servers by hand and gave IT personnel more time to focus on items in the backlog.

The Benefits

Thanks to ClearScale’s support, Pathlight was able to migrate its databases to Amazon Aurora without disruption. ClearScale’s engineers augmented Pathlight’s in-house IT team, allowing the client to focus on important day-to-day tasks, while ClearScale dealt with the larger migration.

Pathlight now has one fully managed, multi-tenant database that scales seamlessly with demand. The AWS platform takes on the administrative burden associated with keeping the organization’s databases online.