Case Study
Multi-Channel Retail Analytics on AWS for a DTC Electric Mobility Brand
ClearScale built a unified AWS data lake consolidating DTC, wholesale, and fulfillment data for a multi-channel electric mobility retailer.
Client Profile
Industry Retail
Competencies
AWS Retail | AWS Data & Analitics
Overview
A US-based direct-to-consumer electric mobility brand operating across DTC eCommerce, B2B wholesale, and third-party fulfillment channels engaged ClearScale to build a unified analytics platform on AWS. ClearScale designed and delivered a complete data lake MVP — including an AWS Control Tower Landing Zone, a multi-stage data pipeline ingesting from three external REST APIs, Apache Iceberg tables on Amazon S3, AWS Glue transformation jobs, Amazon Aurora PostgreSQL data mart, and Tableau Cloud dashboards. The pipeline runs daily in production, surfacing cross-channel Net Sales and SKU reconciliation views that were previously impossible to produce.
Meet Our Hero
The client is an American lifestyle adventure brand and manufacturer of motorcycle-styled electric bicycles, known for celebrity custom builds and high-profile e-bike collaborations. The company sells direct-to-consumer through Shopify, distributes B2B via a wholesale management platform, and fulfills orders through a third-party logistics provider. As the multi-channel business scaled, the inability to see consolidated data across DTC, B2B, and fulfillment systems became a direct constraint on operational decision-making.
The Goal
Build a unified data lake on AWS that consolidates sales and operations data from all three operational systems — DTC eCommerce, B2B wholesale, and 3PL fulfillment — into a single analytics platform. The platform must deliver cross-channel Net Sales reporting, SKU mismatch detection, and inventory visibility through self-service Tableau Cloud dashboards. The full pipeline must run daily on a fully serverless, infrastructure-as-code foundation, with a governed multi-account AWS Landing Zone as the security baseline.
The Challenge
Challenge 01
Three Disconnected Data Sources: Sales and operations data lived across three completely separate systems — DTC eCommerce (Shopify), B2B wholesale (HubSoft), and 3PL fulfillment. There was no unified view of orders, shipments, or net sales. Cross-channel performance analysis required manual extraction from each system separately and manual reconciliation — slow, error-prone, and not scalable.
Challenge 02
No AWS Data Infrastructure: The client had no existing AWS data platform. ClearScale needed to deliver both a governed AWS foundation (Landing Zone with Control Tower) and a complete data lake — ingestion pipelines, storage layers, transformation jobs, and a relational data mart — built from the ground up in a single engagement.
Challenge 03
Diverse Source API Formats: Each source had a distinct API structure and data model — nested JSON from DTC, B2B-specific fields from wholesale, and XML from 3PL. Each required source-specific ingestion logic, schema management, and transformation patterns, all running on a unified pipeline architecture.
Challenge 04
Self-Service Analytics for Business Users: The platform needed to surface actionable cross-channel reporting — Net Sales reconciliation, SKU mismatch detection, and inventory views — accessible to business users in Tableau Cloud without requiring direct database access or knowledge of raw source schemas.
The Solution
ClearScale built the complete platform from AWS account structure through to Tableau dashboards. The full pipeline runs daily in production, ingesting from all three data sources and populating the Aurora data mart.
Step 01: AWS Landing Zone — Control Tower, Multi-Account Governance
ClearScale deployed AWS Control Tower with AWS Organizations: Core OU containing Audit and Log Archive accounts, and Workloads OU containing Dev and Production accounts. AWS SSO was configured with the default Identity Store. Mandatory and strongly recommended guardrails were applied across all OUs. Amazon GuardDuty was enabled across all Control Tower managed accounts with the Audit account designated as GuardDuty Admin. AWS Security Hub was enabled with the AWS Foundational Security Best Practices standard, and 14 CloudWatch Alarms aligned to the CIS AWS Foundations benchmark were deployed to monitor for root account login, unauthorized API calls, IAM policy changes, CloudTrail changes, and other security-relevant events.
Step 03: Data Transformation and Aurora Data Mart
AWS Glue jobs read from Iceberg tables and apply the Relationalize transformation to flatten deeply nested JSON. Transformed data is pushed to Amazon Aurora PostgreSQL with separate schemas per source. Cross-source database views join data across all three source schemas — enabling Tableau dashboards to query clean, consolidated data without business users needing to understand raw API structures. Database schema changes are managed by a Liquibase migration Lambda, with all schema changelogs version-controlled. AWS Step Functions orchestrate the full pipeline — one state machine per data source — triggered automatically daily at 4am UTC.
Step 02: Multi-Stage Data Pipeline with Apache Iceberg
AWS Batch jobs written in Java/Spring ingest data in parallel from all three external REST APIs. The pipeline supports two modes — full load (initial run) and incremental load (subsequent runs) — controlled via AWS SSM Parameter Store. Raw data is written to an S3 RAW bucket in source format, with no transformation at ingestion. AWS Glue jobs convert raw JSON and XML into Apache Iceberg tables on a CONSUMABLE S3 bucket, with the Iceberg Connector for Glue 3.0 from AWS Marketplace handling the table format. Iceberg’s ACID guarantees and schema evolution support are essential for a multi-source data lake where source API schemas evolve over time. After processing, raw files move to an ARCHIVE S3 bucket — enabling full pipeline reprocessing without re-ingesting from source APIs.
Step 04: IaC, CI/CD, and Tableau Dashboards
All infrastructure is deployed via AWS CDK (application stacks — network, database, batch, Glue, Step Functions, Lambda) and Terraform (VPC and networking). Terraform state is stored in S3 with DynamoDB lock to prevent concurrent modifications. The Deployment Runbook documents the full deploy sequence end-to-end. Four Tableau Cloud dashboards were delivered: Net Sales (DTC view), Net Sales (B2B view), Net Sales (3PL Only), and SKU Mismatch — providing the client’s business users with cross-channel analytics for the first time.
The Impact
3 disconnected data sources unified into a single AWS data lake — enabling cross-channel Net Sales analysis and SKU reconciliation for the first time.
4 Tableau Cloud dashboards delivered to business users — Net Sales (DTC, B2B, 3PL views) and cross-channel SKU Mismatch detection — replacing manual cross-system reconciliation work.
100% infrastructure-as-code across AWS CDK (application stacks) and Terraform (networking) — every component reproducible across Dev, Test, and Production environments.
Fully serverless data pipeline running daily in production via AWS Batch, AWS Glue, AWS Step Functions, and AWS Lambda — eliminating idle compute cost between pipeline runs versus a persistent-cluster approach.
Governed multi-account AWS Landing Zone with Amazon GuardDuty, AWS Security Hub, AWS CloudTrail, and 14 CIS-aligned CloudWatch alarms providing the security and compliance foundation for the data platform.
Turn Cloud Chaos Into Clear Results On AWS
Clearscale helps marketing and SaaS companies cut through cloud chaos and get clear results on AWS. If your legacy infrastructure is holding you back and you need a partner to tackle complex, large-scale migration and modernization projects, let’s talk.