The Meter Data Management – MDM – system comprises:
- Specific middleware for data acquisition, data aggregation, data structuring and transformation, so as to feed all functionalities and services in a generic way.
- Functionalities and services for data validation, integration / storage, consolidation and access.
- Generic applications that can be realized in a modular way through the invocation/combination/ orchestration of functionalities and/or services. These applications are still generic but form the basis for those offered to clients, either through parameterization or through client specific instantiation.
In the MDM system we have developed energy data are processed and enriched with other building characteristics and data, such as area, population, etc.
Data analysis is one of the most important parts of the MDM system and is also the system responsible for all calculations and data-mining algorithms, either in batch mode or in real time.
Data mining algorithms and techniques that are used by the MDM system allow users to analyze data from many different angles, categorize it and summarize the relationships identified. In essence, data mining is the process of finding the correlations or patterns among dozens of fields in the existing large relational energy database. A recursive algorithm is executed in order to analyze the results through time and correlate them with other external variables (temp, humidity etc).
The process of finding hidden information included in data has different components, including knowledge extraction, information discovery, data mining and knowledge discovery from energy databases. In particular, data mining includes other activities, such as data selection, checking, cleaning, preparation, pattern presentation and knowledge refinement and visualization.
In the MDM system a load profiling method is applied and a specific data mining technique on agent grid clustering is used for the processing of the data. The objective of this design is to determine the best techniques to be used, in order to reach the energy saving target that has been set.
For effective load profiling and data mining, the following steps are followed:
- Set specific energy Key Performance Indicators (KPI) from the metering grid and the load curve mathematics.
- Categorise and compute specific objective functions, comprising specific KPIs.
- Perform a weighted version of the recursive analytics algorithms, based on a distributed agent structure routed on the cloud.
- Store the numerical values in matrices, called Energy Relevance Matrices (ERM) and use numerical linear algebra to find correlations and hidden patterns.
- Use the results of the Energy Relevance Matrix analysis together with the output of the agent results vector in order to identify cross-hidden correlations, periodicity and peak trends on the energy load curves.
Energy data become available through APIs in JSON, XML, plain text, HTML and CSV format. The most appropriate format is JSON due to its flexibility and the smaller size of its files. The available API methods include: daily data in one-hour intervals; monthly data in one-day intervals; yearly data in one-month intervals.