microTERRA Builds Robust IoT Capability to Monitor Water Quality Across Thousands of Remote Devices

Executive Summary

microTERRA is a Mexico City-based company that uses biotechnology to generate clean water and sustainable feed ingredients from aquaculture wastewater. The company develops microalgae-based wastewater treatment systems that convert water pollution into high-value microalgae ingredients for animal feed, such as protein. microTERRA helps fish farmers establish circular economies that benefit the surrounding agro-industry in many ways. microTERRA’s novel system improves animal feed quality, increases clean water access, reduces freshwater consumption, decreases greenhouse gas emissions, and more. Its systems are tailored for LATAM environmental conditions (sunny and warm regions) and operate onsite. microTERRA integrates the Internet of Things (IoT) to monitor the performance of systems and assist farmers with operations remotely and in real-time.

microTERRA’s technology also has implications outside of in-land aquaculture. The company has the potential to treat wastewater rich in manure from other animals, including cows, chickens, and swine. After partnering with ClearScale, microTERRA has the IoT capabilities it needs to automate microalgae growth monitoring and enhance the value of its services for fish and livestock farming all over the world.

microTERRA’s mission is to clean the contaminated water on-site, eliminating downstream pollution, and transforming animal manure into valuable ingredients for animal feed.

The Challenge

microTERRA wanted to harness the power of the IoT for its microalgae-based wastewater treatment system offering. The business built Raspberry Pi devices that could independently monitor water quality and performance of microalgae on-site. Data and log files were aggregated and uploaded to Google Drive using API requests. However, updating code and offloading sensor data required significant manual effort.

microTERRA saw an opportunity to automate aspects of this process, including:

• Device-to-cloud connectivity
• Device management
• Data ingestion and processing

The company’s leaders envisioned a minimum viable product (MVP) on the Amazon Web Services (AWS) IoT stack that included a multitude of capabilities.

With AWS IoT Core, microTERRA hoped to gather data from remote Raspberry Pi devices, sync with shadows during intermittent connectivity, and securely transmit field information to the cloud.

The solution also needed to take advantage of two types of analytics: real-time and batch analytics. In addition, microTERRA wanted users to be able to receive notifications instantaneously about next actions to maintain and operate the wastewater treatment system, environmental anomalies and threshold exaggerations.

Adding complexity to the project was the fact that microTERRA’s existing Raspberry Pi devices could not connect continuously to the internet. As a result, every device had to maintain internal storage space and be able to distinguish between new data and data that had already been sent to the cloud. The team also wanted its devices to utilize exponential backoff to properly handle network failures, which meant a de-duplication mechanism had to exist on the AWS side.

To implement these features, microTERRA needed to integrate with AWS IoT Core, possess an ability to control all devices from a central point, and enable over-the-air (OTA) updates. Finally, all devices had to maintain certain security standards and ensure reliable data transfer at all times.

Unfortunately, microTERRA didn’t have the expertise to execute such a complex IoT project of this magnitude. The organization decided to look for a professional cloud services provider with extensive big data and IoT implementation experience.

Enter ClearScale.

In ClearScale, MicroTERRA had the partner it needed to bring its IoT vision to life and extend the value of its aquaculture solution even further for local fish farmers.

The ClearScale Solution

ClearScale’s expert cloud specialists began by gathering all of microTERRA’s business requirements. With this information, ClearScale mapped out a solution consisting of four key elements built with AWS platform:

• Data ingestion layer
• Data storage layer
• Data processing and data query layers
• Data visualization layer

Data Ingestion Layer

microTERRA needed a way to efficiently aggregate water quality data from all of its Raspberry Pi devices. While connecting a small number of IoT sensors to the AWS cloud is relatively easy, managing hundreds or thousands of devices is complex and time-consuming.

To address this challenge, ClearScale’s experts developed a device provisioning tool that connects devices to AWS IoT Core, and automatically handles certification management, Debian package device registration, and connection establishments. With this tool, microTERRA had a secure data ingestion layer that could support millions of IoT devices simultaneously. No matter how much or how quickly the company grows, ClearScale’s provisioning tool can scale accordingly.

Data Storage Layer

microTERRA also required a flexible data storage layer that could maintain field data without incurring exorbitant fees. To fulfill this requirement, ClearScale set up a data storage layer with three distinct components.

The team implemented Amazon S3, an object storage service, to collect raw, unprocessed data from microTERRA’s Raspberry Pi devices. IoT data is stored as parquet files and partitioned as needed.

ClearScale used Amazon DynamoDB, a NoSQL database that can scale to any size, for processed data. The DynamoDB tables store the information that is used to populate microTERRA’s analytical dashboards.

Finally, ClearScale set up an Amazon RDS Postgres database to store the structure of the role model used for the Web UI application. The Postgres database also collects technical info, such as alarm histories, so microTERRA can evaluate past water pollution levels.

Across these three data storage methods, microTERRA has a cost-effective, cloud-based infrastructure in place for maintaining all critical application data.

Data Processing and Data Query Layers

Because microTERRA wanted to execute both real-time and batch analytics, ClearScale decided to use a lambda architecture.

To enable real-time analytics, ClearScale implemented Amazon Kinesis, a powerful AWS service that simplifies the collecting, processing, and analyzing of real-time streaming data. Analytics are executed using SQL window queries.

On the batch analytics front, ClearScale implemented the interactive query service, Amazon Athena, for data stored in Amazon S3. Along with Amazon DynamoDB, Athena is instrumental in populating the company’s data reports.

Together, Amazon Kinesis and Amazon Athena give microTERRA the ability to process and query IoT data rapidly, while only paying for computing power used. With these tools, the company’s users can keep up with device readings across a vast geographic footprint.

Data Visualization Layer

Finally, microTERRA wanted to be able to visualize sensor data to better understand the effectiveness of its aquaculture systems.

ClearScale implemented a data visualization layer with Amazon QuickSight, a business intelligence service that offers per-per-session pricing. The feature enables users to easily publish and share analytical insights, including those generated by machine learning programs.

The ClearScale team also developed a separate Web UI application for microTERRA users. The solution can display device maps, provide quick access to numerous dashboards, and carry out certain management tasks.

Now, microTERRA has an intuitive user interface that helps users quickly identify key analytical insights related to its aquaculture wastewater treatment systems.

Architecture Diagrams

DataFlow diagram:

Batch analytics pipeline:

Streaming analytics pipeline:

The Benefits

Thanks to ClearScale’s help, microTERRA now has a sophisticated and scalable IoT capability that is powered by multiple AWS cloud services.

microTERRA can easily manage its fleet of Raspberry Pi devices and monitor water quality across numerous project sites. The team no longer has to implement code updates or gather data manually, enabling individuals to focus on higher-value work.

microTERRA can analyze data in real time or in batches to assess water quality from different angles, as well as build insightful dashboards. Additionally, only one engineer is needed to support all devices, which means microTERRA spends less money on device management overall.

The biotech firm can also get to market more quickly with its flexible cloud-based data layers. microTERRA is able to execute complex computing tasks with tremendous efficiency, thus increasing the ROI of its aquaculture engagements.