Forecasting Linked Data Event Streams on the Internet of Water

Author(s): Samuel Van Ackere

Originally published on Towards AI.

Applying Facebook’s powerful forecasting model to Linked Data Event Stream IoT sensor data for improved water management

As our world becomes increasingly data-driven, the ability to accurately forecast future events has become crucial for businesses and organizations. Linked Data Event Streams are known as a powerful standard to represent and share big data in real time.

In this article, I explore the potential of using Prophet for forecasting Linked Data Event Streams (LDES), specifically focusing on an Internet of Water use case.

Linked Data Event Stream

A Linked Data Event Stream (LDES) is a continuous flow of immutable objects, such as version objects, sensor observations, or archived representations. Each object contains information about an event or change of state that originates from a system that continuously generates data. The use of LDES for Internet of Water IoT sensor time-series data allows for integrating multiple data streams from different sources, making it possible to connect and analyse data from a wide range of IoT devices.

For more information on what an LDES is, please read this article:

Linked Data Event Streams explained in 8 minutes

What is a Linked data event stream, and how can it help you make your dynamic data accessible?

medium.com

Before forecasting the LDES stream, I need to ingest this into a database. After this materialization, it becomes possible to forecast the stream’s data in one batch.

This is how to ingest an LDES in a TimescaleDB or PostgreSQL:

Linked Data Event Streams and TimescaleDB for Real-time Timeseries Data Management

How to consume a Linked Data Event Stream and store it in a TimescaleDB database

pub.towardsai.net

To show the potential of Machine Learning on Linked Data Event Streams, I showcase a Prophet forecasting model on an Internet of Water (IoW) LDES. Immediately one of the benefits of LDES becomes apparent; in addition to the fact that you are always in sync with the most up-to-date data via an LDES, the LDES also contains historical data. As a result, it becomes possible to immediately harvest all the necessary historical data to train the offline forecasting model.

Underneath, a part of the historical data of the IoW sensor data is visualized (temperature and conductivity).

Forecasting IoW data

I will use the Prophet forecasting model to forecast the Internet of Waters (IoW) LDES. Prophet is a powerful forecasting model developed by Facebook that is particularly well-suited for time-series data. It uses a combination of linear and non-linear methods to model trends and seasonality in data, enabling it to handle both short-term and long-term forecasts. The model can also handle missing data points and outliers, making it more robust to real-world data.

When applied to time-series sensor data, Prophet can predict future events such as water usage, temperature, and equipment performance. These predictions can improve operations' efficiency, safety, and sustainability by enabling organizations to anticipate and prevent potential issues. Additionally, by incorporating data from multiple sensors, Prophet can be used to predict the interactions and relationships between different data streams.

Underneath is a code snippet of the prophet forecasting model trained on the IoW LDES and generating a prediction.

This code generates a forecasting of the time series for three consecutive days. The benefit of using Prophet model is that it is easy to add prediction intervals together with the forecasted data. This is a good way to provide uncertainty estimation in your forecasting.

Full code

To replicate the data flow in this article, please go to the ML-LDES server. It describes how to set up the dockerized PostgreSQL/PostGIS, PgAdmin, and Apache NiFi, after which the data flow can be started using the supplied Apache NiFi setup file.

GitHub – samuvack/ML-LDES-server: The Machine Learning (ML) server for Linked Data Event Streams…

The ML-LDES server was built by the Flemish Smart Data Space project to predict, analyse and find relationships on…

github.com

Wrapping Up

A Linked Data Event Stream is considered the core API of fast and slow-moving data. It is a data event stream of a group of immutable objects described as machine-readable RDF (such as sensor observations, address registers or financial data).

In this article, I tried to use an offline forecasting model Prophet on such an LDES, explaining from start to finish how to set up this data flow.

If you like what you read, be sure to U+2764️ it — as a writer, it means the world. Stay in touch by following me as an author.

Contributors to this article are ddvlanck (Dwight Van Lancker) (github.com), sandervd (Sander Van Dooren) (github.com) at Flanders Smart Data Space (Digital Flanders, Belgium). In a rapidly changing society, governments need to be more agile and resilient than ever. Digital Flanders realizes and supervises digital transformation projects for Flemish and local governments.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI