Problems

Have you ever wondered whether it is possible to do the following things?

• Using SCADAConnect to compute WaterGEMS every X minutes and automatically export results using custom logic.
• Automatically post-process computed results to produce a report to custom specifications.
• Convert computed results into a format that can be readily consumed by a 3rd party system such as Power BI or other web dash-boarding tool.
• Execute some additional calculations then update model data based on those results.
• Do a statistical analysis on computed results and trigger some kind of action based on the results.
• Anything else that can be defined as acquiring some data, acting upon it, then producing some kind of output.

Solution

Export Results to SQL Server

An easier-to-use tool is now available as part of the standard WaterGEMS installation (starting with version 10.03.02.75 and higher), which can be used to configure hydraulic model results to automatically export to a SQL server database. In supported versions of WaterGEMS you can find the documentation on how to use the tool in the WaterGEMS installation folder (default: C:\Program Files (x86)\Bentley\WaterGEMS) in the PDF file currently named Haestad.DataIntegration.Driver.pdf (this name may change in future versions). The PDF can also be found in the Documents section below.

An example dataset is also available at the bottom of the "Sample Files" section below.

Export Results to CSV or custom capabilities

Users who are familiar with WaterObjects.NET will be aware that it is and has been possible for a long time to write .NET code against the WaterObjects object model to carry out tasks such as reading and writing model data and also reading model results. With regards to the kind of use cases described above, what has been lacking in the past is an easy way to write code that can be coupled with WaterGEMS in a way that can execute automatically and also the provision of generic support for doing the types of things that data processing tools might need to do, such as command line processing, interacting with files and directories, file archival, working with databases, providing configuration support and logging operations.

In the following series of documents you will find a description of a framework called the Haestad.DataIntegration Framework (HDI). Included in these documents are a description of several off-the-shelf functionalities (built on the framework) that enable automatic, configurable, conversion of WaterGEMS energy costs and hydraulics results into comma separated values (CSV) format. Such support could be useful if wanting to automatically publish results data for key assets to other systems such as Bentley AssetWise Operational Analytics, Power PI, OSI PI, or other real-time/dashboarding type system.

Furthermore, you will find documents that describe how using this framework it is possible to write software components that have the following traits:

1. Able to convert data from one format to another. (e.g., model results to CSV, model spatial data to SQL server, demand data to model, etc.).
2. Support user configurable behaviors.
3. Conform to a standard specification, thus allowing for a standardized and robust boot-strapping process.
4. Support different levels of logging (e.g., errors, warnings, info, debug etc).
5. Easily extensible and re-usable.
6. Can be executed automatically either out-of-process (by the HDI driver) or in-process (by a compatible WaterObjects application (e.g., SCADAConnect Simulator)).
7. Only limited by your imagination in terms of custom capabilities.

Documents

The series of documents that describe this framework are listed below. The first document is a good place to start as it provides a bird's eye view of the framework, plus describes how the additional documents relate to the various pieces of the framework. For instance, if you are interested in publishing results to CSV format after a SCADAConnect Simulator auto-compute, then you may be able to jump right into the "WaterGEMS to CSV User Guide". If, however, you have more specific requirements that may require some kind of customization, you might be better off starting with the "Publishing Hydraulic Modeling Data to 3rd Party Systems" document. The last two documents talk specifically about how to hook custom post calculation logic into SCADAConnect Simulator, though this is not a requirement for utilizing the framework.

Note: to download these files, you will need to be signed in.

DataIntegration Framework Overview

Publishing Hydraulic Modeling Data to 3rd Party Systems

DataIntegration User Guide

DataIntegration User Guide (In Process)

WaterGEMS to CSV User Guide

WaterGEMS to CSV User Guide (Batch Processing)

SCADAConnect Post Calculation Batch File User Guide

SCADAConnect Post Calculation Data Exchanger User Guide

WaterGEMS to SQL Server User Guide

Sample Files

In addition to the above documents, several sample files are available as follows:

Watertown (7z File) - (Used with WaterGEMS to CSV User Guide)

BatchSample (7z File) - (Used with WaterGEMS to CSV User Guide (Batch Processing))

SampleDataExchanger (Zip File) - (Used with Publishing Hydraulic Modeling Data to 3rd Party Systems)

SCADAPostCalculationDataExchanger (Zip File) - (Used with SCADAConnect Post Calculation Data Exchanger User Guide)

WaterGEMS to SQL Server Sample - (used with WaterGEMS to SQL Server)

Notes

Some of the functionality described in this document is available with WaterGEMS versions as early as WaterGEMS 10.01.XX.XX, however, some of the most recent functionality, including in-process data exchanger support in SCADAConnect Simulator, require the latest Update 2 of WaterGEMS (10.02.01.06) in addition to the latest patch set (Patch01 or higher - please contact Technical Support via the Forum or Service Request Manager).

See Also

SCADAConnect Simulator for WaterGEMS (V8i SELECTseries 6 and CONNECT edition)

WaterObjects.NET FAQ