How to increase model performance / reduce run time (recommended minimum system requirements)

Product(s): WaterGEMS, WaterCAD, HAMMER, SewerGEMS, CivilStorm, StormCAD, SewerCAD
Version(s): CONNECT Edition, V8i
Area:  Calculations

Problem

My model takes a long time to run or is sluggish when navigating the Drawing Pane, generating reports, etc. What can I do to speed this up?

What are the minimum recommended system requirements? (machine specs)

Background

Although practically any computer running Windows 7 or greater will run WaterCAD or WaterGEMS as of this writing (see the System Requirements in the Readme.pdf file located in the installation folder and Operating System compatibility here), model performance is highly dependent on the model size and complexity. Sometimes the amount of time it takes to compute may be appropriate due to how large or complicated the model is. For example, if you have a model with hundreds of thousands of pipes with a lot going on with controls and things like pressure dependent demands, variable speed pumps, hydropneumatic tanks, etc. However, there are a few factors to consider, and there are other "performance" issues that may arise, such as taking a long time to open, save, navigate, etc.

Solution

Model computation: Excessive Timesteps (WaterCAD, WaterGEMS)

Ensuring that the model is 'clean' and that no extra intermediate timesteps are being unnecessarily generated tends to be the best first step at resolving issues with model performance. You may want to graph all of your tank's "percent full" attribute to ensure that none are becoming full or empty during the simulation, as this is the most common cause for intermediate timesteps being inserted (as it can cause the adjacent pipe to close and open rapidly). A quick way to do this is to navigate to Edit > Select by Element > Tank > right click > Graph.

You can also control the amount of result output using the "Reporting Time Step" calculation option. If you know that a large amount of results need to be generated, you can reduce the amount that needs to be saved to disk.

Make sure your calculation timestep (under calculation options) is appropriate. The default is one hour, with a slightly smaller timestep such as 0.1 hours sometimes appropriate depending on the situation.

Model computation: Stability (WaterCAD, WaterGEMS)

Stability problems in a WaterCAD/WaterGEMS model can also cause it to take longer to compute. Instability can cause more intermediate timesteps to be generated (per the item above) but can also cause an excessive number of trials to be required for each timestep. The Calculation Summary will show you the number of trials needed for each timestep, to quickly identify if this is a culprit. You may also notice a red icon next to timesteps that have a large number of trials - in those cases, the network may be unbalanced. This means that the solver is using an excessive number of calculation trials to try to settle on a balanced solution. See the link below for help with troubleshooting an unbalanced network. Once the model is stable, performance should increase.

Troubleshooting "Network Unbalanced"

Transient duration, simplification, calculation options (HAMMER)

For a transient simulation in HAMMER, you can speed up the performance of calculations by shortening the duration, simplifying the model and adjust the reporting-related calculation options. See details here: Improving performance of transient simulations (recommended reporting calculation options)

Opening, saving, entering data, generating reports, panning/zooming, etc

Model storage location

If the model takes a long time to open or save, consider where it is stored. Storing a model on a network drive / shared network folder is generally not recommended due to relatively large amount of time it can take to copy all of the files over the network when opening or saving (especially with large models which can be several hundred megabytes in size). This also increases the chances of data corruption, if the network experiences an interruption in the middle of saving. if you must store the model on a network drive to share with colleagues, consider working locally (save a copy on your hard drive) and then when done, copy the main model files to the network location (.WTG/.WTG.SQLITE or .STSW/.STSW.SQLITE for example).

Furthermore, when saving a model, many output files can be saved, which store the hydraulic results. Separate results files are stored for each scenario and for different features such as flushing and fireflow. These can accumulate and need to be downloaded from the network drive when opening the model. So, if the model starts to take a long time to open, consider deleting these output files (in case some are from scenarios that you do not look at anymore). Only the .WTG and .WTG.SQLITE file need to remain in the folder, as all other files are output files or will regenerate when re-opening the model. See: Many output files are being saved

See also: Opening an existing model takes a long time

Large and/or complex background layers

If you have large background layers attached, consider simplifying them if possible, take advantage of other platforms (such as WaterGEMS for ArcMap), or be sure to use a copy of the background layers stored on your local hard drive.

Excessive Element Symbology or background layers

If performance is sluggish when panning around the model or generally working with the model, check if you have unusual or excessive element symbology or background layers. For example in one case it was found that extensive selection set-based annotations and color coding were set up, significantly impacting performance. If background layers and element symbology are reasonable, try closing the model, delete the .DWH file (same base file name and location as the main model files), then re-open the model. This file stores information on how the network appears in the drawing pane and will be regenerated when the model is re-opened. If that does not help, try opening the model database (SQLITE) file alone, which is done by selecting "database files" from the "files of type" dropdown in the file open dialog. This will omit any element symbology, as a way to narrow down the problem.

Background layer performance can also be impacted by the graphics hardware on your computer. Ensure that your dedicated graphics hardware (if available) is being utilized, update your video drivers and try toggling the Accelerated Redraw option per this article: Updating video drivers and troubleshooting video hardware issues

Large number of catchment polygon vertices

In a storm/sewer model with a large number of catchments, sluggish performance during zoom/pan could be due to the level of detail in the borders of the catchments. Check the properties of your catchments and look at the "collection" number in the "geometry" field at the top, which indicates the number of vertices / line segments that make up the catchment polygon. Note that this field is not available in Flextables or ModelBuilder so a table cannot be sorted - consider using the export button in the catchment flextable to export to shapefile, then review in a GIS application such as ArcGIS or QGIS.

In previous cases it was found that catchments had thousands of vertices, causing the overall model to attempt to render millions of points, causing a slowdown in performance. In such cases if the catchments were imported via ModelBuilder, consider reducing the granularity of the catchment vertices in an external application such as ArcGIS or QGIS, then use ModelBuilder to update the geometry (or delete the catchments and re-import with ModelBuilder.) In the ModelBuilder configuration, you can uncheck the "Create" and "Remove" options leaving only the "update" option and sync-in, which will update the vertices without changing anything else.

Another option would be to uncheck the Catchment layer in Element Symbology, to make them disappear. You would still need to assign the catchment outflow node and configure other parameters, which could be done from the catchment Flextable. If performance is still slow, try saving the model, closing it, deleting the model's associate .DWH file, then re-open.

You can also check your computer specifications (in particular the video hardware, CPU and memory), and check if other running applications/tasks are taking up your resources (for example antivirus scans).

Your computer's video hardware can also impact performance when panning/navigating the drawing pane. See further below.

Large number of customer meters, taps, laterals or isolation valves

If these elements are used, it can add a lot of extra detail to the model plan view especially for larger models when zoomed out. In this case, consider Using Zoom Dependent Visibility to cause elements to disappear at certain zoom levels

Use of Virtual Machines

If performance is sluggish when panning/zooming/moving around the model while using a virtual machine, you may be limited by the hardware capabilities of the virtual machine. See: Using OpenFlows products with Citrix or other Virtualized environments

General: Computer Hardware

Consider your computer's hardware. A faster CPU, more RAM and a faster hard drive (such as a solid state drive instead of a mechanical hard disk drive) can certainly increase performance. The system requirements in the documentation are a good start, but larger, more complex models will require more resources to achieve optimal performance. One reason why there is generally no "recommended" computer specs for the OpenFlows products is that it is highly dependent on the size and type of models and frequency of calculations. For example, if an engineer only works on relatively small models, then even the "minimum" specifications listed in the Readme file should be fine, as the run times will be on the order of seconds.

CPU: In most situations the clock rate (ex: 3.1 Ghz) of your computer's CPU has the most impact on how long a model takes to computer. WaterCAD and WaterGEMS currently utilize a single core for the majority of hydraulic computations*, a faster clock rate with a smaller number of cores will typically provide better performance than a slower clock rate with a larger number of cores. 

*Note: although a single core is currently used for a majority of hydraulic calculations, it should be pointed out that WaterGEMS/WaterCAD as a product can and does make use of multiple CPU cores. There are many parts of our applications that do work on different threads and having multiple CPU cores can improve the performance of the product in general. Also, automated fire flow calculations do support using multi-core and it is possible that other areas of the computational engine will support multi-core processing in future. So whilst multi-core may not make a huge difference right now, if you are buying new hardware then thinking about future proofing does have some merit.

Also, some hydraulic model runs may be more hard drive (storage) intensive than others, and performance may rely a bit more on the speed of the hard drive and possibly the memory. For example, consider a model that is set to save results for all times for a small increment and long duration, but with a model that is relatively not as calculation intensive (for example small number of controls, fairly easy to solve network). Contrast that to a model set to only save a small number of timesteps but with a large, complex network. Performance of the former may rely more on hard drive and memory, and a solid state drive would likely be best (see below). 

RAM: The amount of physical memory (RAM) in your computer typically will not impact the performance of your hydraulic models, unless the model is large enough to approach your memory capacity. Large background layers are one example of something that can cause excessive memory usage. Check the Details tab of your task manager (right click task bar > Task manager) to see how much memory the program is taking up.

Ensure that you use the 64-bit version of WaterCAD/WaterGEMS in order to utilize all of your computer's memory. Most computers and operating systems are 64-bit and the 64-bit version of the program will be installed by default. To confirm, go to Help > About and check next to the date and version in the lower left corner. If you're using the 64-bit version, you will see "64-bit".

For more information on concerns related to RAM, see: Memory errors when performing certain tasks

Hard Drive / Storage: The type and speed of your computer's hard drive can impact performance when the program reads information from it (the model files) and writes information to it (model files and results). A traditional mechanical Hard Disk Drive (HDD) generally has higher capacity but lower speed. Because these types of drives involve a mechanical, spinning disk, a latency is introduced. Contrast this to a Solid State Drive (SSD) where information is written to and read from memory chips, where no mechanical latency occurs. So, installing WaterCAD/WaterGEMS on a SSD can increase performance of opening and working with the program. Storing hydraulic model files on a SSD can increase performance as far as the time it takes to open the model and the time it takes to write results (representing a portion of the total time to compute a model).

The hydraulics and hydrology products often write a lot of information to disk, especially for large models and having an I/O device that supports fast throughput can markedly improve the user experience.

Video Card / Graphics hardware: Model run times are typically not impacted by video / graphics hardware, but can have an impact on the responsiveness of the drawing pane. For example, when working with a large model with many background layers and/or complex symbology, panning, zooming, etc can potentially be sluggish. In these cases, you graphics hardware (display adapter) can impact performance. Discrete video cards have better performance when compared to integrated video hardware such as "Intel HD" (integrated into Intel CPUs). To check your video hardware, navigate to Windows Control Panel > Device Manager > Display Adapters. If you have both a discrete and integrated display adapter, ensure that WaterGEMS/WaterCAD is using the discrete video card. To check, look at the discrete video card's control panel. For example for NVIDIA discrete video hardware, you would typically right click on your desktop > "NVIDIA Control Panel" > Manage 3D settings > Global Settings > Preferred Graphics Processor > Higher-Performance NVIDIA Processor. 

*Note that a high-end discrete video card on a Desktop computer can sometimes require a higher wattage Power Supply (PSU).

Output file location

Ensure that you're saving output files to the right place. Under Tools > Options > Project, ensure that the "specify custom results file path?" option is either unchecked, or is set to save results to a suitable location, such as a local hard drive with adequate storage space available, or a solid state drive.


Database Size

The size of the model database could have an impact on performance. It is generally a good idea to Compact the model database periodically. If possible, ensure that the model and Windows temp folder is located on a solid state hard drive, rather than a spinning hard disk drive.

Recommended
Related