I-NERGY Data Services
The I-NERGY architecture, as described into a previous article, of the blog post section of the I-NERGY official website, includes the Data service layer as middle layer and gateway between the project data providers and the services/tools/digital twin and modules developed in the context of the Energy Analytics Service & Application and AI models Training suite project layers.
Data Service layer collects, transforms, stores and /shares data in order to be further processed by upper layer components.
Project data comes from different data-sources: the main contribution is given by the nine project pilots and from potential external data sources such as EC public datasets, weather datasets, climate data, assets (PV, building, EV charging station…) metadata and every kind of data useful for the project scope.
Data service layer consists of different modules which, through their interactions, contribute to the collection and management of project data. An architectural overview is given by figure below.
Main modules involved in the data ingestion pipeline are:
- Interoperability module, responsible for the management off all the connectors developed to integrate inside the platform the data coming from the different project data sources. The module can collect data in two ways, retrieve data and receive data based on the type of data source.
- Homogenisation Module, directly connected with the Interoperability module sharing the same environment and software core components. It has in charge the data harmonisation in terms of data pre-processing such as data cleansing, anonymization, outlier suppression and data formatting to a specific project common data model and ontology, to prepare data for the storing/streaming.
- Data Streaming module is in charge to stream ingested data to the I-NERGY "Analytics Services & Application" and to the "AI models training suite" layers. The data streaming is in near real time with low latency for all the applications that need to preces this type of data.
- I-NERGY Data storage represents the project data base to store harmonised data to be available to all the project services and tools that will use them.
Previous modules are under the umbrella off other two cross functional modules for whole project architecture that are:
- Hybrid data sharing B2B Cross-stakeholder Trusted off Chain data sharing and re-use is a module to allow the trusted data sharing within the whole project platform.
- The Security Framework made by a series of security techniques and tools to ensure security aspects within the I-NERGY ecosystem.
Modules interaction in data management phases are summarised in figure below.
