Building Scalable and Safe Analytics Reporting Service by Leveraging Serverless Architecture

Executive Summary

ReachLocal, a division of Gannett, is an online marketing and advertising solution provider for small, medium, and large businesses in the United States, Europe, Asia, and Australia. It was the first company created specifically to help businesses solve the unique challenges of online marketing advertising to clients in local regions.

The company provides Internet and mobile Internet marketing solutions, and has spent over a decade building digital marketing services focused exclusively on local customers. ReachLocal works with clients to understand their businesses, develop digital marketing strategies, and manage the solutions that offer the best results from online marketing.

By analyzing over a million local campaigns, ReachLocal has gathered powerful, actionable insights, and its experts collaborate with clients to help drive new business growth. To date, the company has delivered over 200 million leads to local businesses worldwide.

The Challenge

In 2017, ReachLocal’s advertisers required API type integrations for all campaign related metrics and reporting into the advertisers CRM/ERP systems.

The internal team had a build a Analytics Reporting Service (ARS) to consolidate the advertisers campaign performance reporting. This was however limited to the advertiser running reports and outputting CSV/Excel files for ingestion to the CRM/ERP systems, not ideal.

The Analytics Reporting Service (ARS) was originally architected to deliver internal reports and couldn’t provide functionality for external reporting data consumption in a safe and secure way.

To address this strategic requirement, ReachLocal decided to architect a solution of moving Analytics Reporting Service to Amazon Web Services (AWS), then expose their Campaign performance reporting metrics APIs to secured authenticated Advertisers using AWS API Gateway. Acknowledging the business needs and lack of seasoned in-house AWS expertise, the VP of Media Engineering Raj Sudra reached out to ClearScale.

The ClearScale Solution

Client Reporting Application

ClearScale team met with Reach Local, went through their requirements, and clarified work sessions. Using a serverless setup, the team deployed a Client Reporting Application that includes an authentication and authorization layer that separates data per customer. This provides access to specific reports, but only for data that belongs to a particular customer.

Here’s an example of the typical data flow inside the Client Reporting Application:

1. Campaign performance data and metrics are obtained from publishers, including Google, Bing, Yahoo, and Yelp, as well from phone calls, form submissions, clicks on ads, text messages, and other leads generated from advertising campaigns.
2. MySQL database is updated with this operational data across the ReachLocal ecosystem on a daily basis; aggregation jobs sum up all metrics.
3. All client center-required tables are replicated into a Maria 10 database, which is then used by a Reporting Analytics Service (ETL process) that feeds the client center application.

New Reporting Architecture

ClearScale team configured AWS APIGateway for ReachLocal customers, which can authorize request and give access only to authorized reports. AWS APIGateway provides correct HTTP responses codes also.

Lambda gets information from AWS APIGateway, validates request, and proxies it to ReachLocal API.

We configured AWS CloudWatch to store logs about requests and responses. AWS CloudWatch triggers send information to ReachLocal Sumologic account via AWS Lambda.

All components of the new data pipeline can be instantly scaled up or down as the volume of reporting data changes.

The new reporting architecture also makes the Analytics Reporting System (ARS) service more scalable. ARS will run within a Docker container, making it easier to deploy and scale services within AWS. In addition, ARS will be able to be hosted by AWS Elastic Container Service, which will make it easy to support demand from reporting clients.

Below is a diagram of the new reporting architecture:

Infrastructure Automation and Continuous Delivery

We used CodePipeline and CodeBuild to implement continuous delivery for Lambda functions. In this solution we push changes to Github repo for an AWS Lambda function. This will trigger the deployment pipeline (created in AWS CodePipeline) which will then update the Lambda function.

See a diagram below:

Infrastructure

We used troposphere library to make python based scripts for dynamic infrastructure automation:

• Deploy/Manage API Gateway
• Deploy/Manage Lambda Functions
• Deploy/Manage Continuous Delivery pipelines

The Benefits

The new Client Reporting Application allows ReachLocal to provide their data via API to their clients in a safe manner. And the new reporting architecture delivers the benefits such as:

• Open architecture, which allows software, media, and business systems to contribute to a set of shared reporting services
• Flexible reporting, with APIs that can be customized for specific data source and report type
• Organizational scalability, where each new API reporting service can be worked on by independent groups of engineers, so more can be added as needed without interfering with each other
• Scalability for each API reporting service, depending on the size and complexity of the report, or the demand on the individual reports