I am facing failure during running SewerGEMS model, the notification does not explain the exact problem, it is not specific to enable us to find the problem (as in the below screenshot).
The model shared via Bentley secured link, I was trying to run scenario 10 years (15 min.), and this issue is applicable for all scenarios when running the Explicit Enging.
Regards,
Mohamad Azzam
Hi Mohamad,
It appears that the EPA-SWMM solver itself is failing partway through the simulation, with the following error:
Access violation in module execRouting at step ___, hour _ --- execution halted.
I could not locate a previous instance of this error occurring before, but I suspect it may be occurring due to the model experiencing extreme conditions or extreme instability. I see that you are using a 15 minute storm with a very sharp peak of over 250 mm/hr, which may be causing such extreme conditions.
If you change the solver compatibility calculation option to use the EPA SWMM solver instead of the Bentley Enhanced SWMM solver, the model computes, but this is likely not an option for you since you are using some features that only the Bentley-enhanced solver version supports, like HEC-22 inlets and HEC-22 structure losses.
I did find some sensitivity (as far as which timestep it fails at) to some of the calculation options but was unable to find a set of options that resulted in a successful run. For the data entry, here are a few things I noticed which you may want to look into, but may or may not be related to the issue in question:
In the meantime, I have reported your model to our development team (reference # 788333). If the above does not help and this is preventing you from continuing with your work, please let us know and we will escalate the defect.
As a side note, I noticed that your SQLITE file is over 7 GB, due to about 56 million Change Tracking records. I recommend using the archive feature to reduce the model file size: SQLITE database size grows very large due to Change Tracking
Jesse DringoliTechnical Support Manager, OpenFlowsBentley Communities Site AdministratorBentley Systems, Inc.
Dear Jesse,
Thank you for the massive investigation into the model and your support as usual do. here are my clarifications;
Jesse Dringoli said:There are numerous wetwells not connected to any pumps, and no controls to prevent them from becoming full or empty.
That is only just to reserve the storm and then allow for certain allowable discharge.
Jesse Dringoli said:here are ponds with no control structure on the attached pond outlet node. Generally using a composite outlet structure tends to provide better stability. Examples: POS-3, POS-5 (this one is missing its ground elevation, too)
Yes - I am aware of that and was intended later for the tuning of the model.
Jesse Dringoli said:The layout around outfall O-4 suggests that it discharges into "Pond shp-1" but it is set to free outfall. Did you mean to set the outfall to Boundary Element and connect it to the pond?
the outfall there is for testing the hydrograph, however, it will be connected to the in other scenarios to the pond.
Jesse Dringoli said:There are some areas with a large number of inlets connected to a single manhole via lateral conduits. For example MH-2-14-3 has at least 20 attached conduits. Is there really a manhole structure with 20 pipes connected to it? Consider modeling the pipe layout to match the real system and avoid a very large number of pipes connected to a single node, if possible.
The inlets in the model are matching the actual location where will be built, the actual connection of these inlets will be as Y connection to the pipes and manhole. I know that can be done by using Tap to the pipe but the way it is connected to the manholes is just for the purpose of modeling.
I hope you can find the solution for this defect in order to finalize the model urgently, as this model is built in the way which the client asked to do. we used the same concept in other smaller models without any issues, even this one was running well but after we increased the inlets numbers and manholes we faces this issue.
PS: as you noticed we are now working on big models and that will continue for the future to use the inlets in all our models as a new regulation from the municipality, accordingly the run of each scenario takes a lot of time, is that can be the investigation and improved by the Bentley developer to enhance the run time? that happen in spite I am using a laptop with high-end specs. seems the program does not take the graphic card benefit during the run. I hope this can be considered.
Mohamad Azzam said:The inlets in the model are matching the actual location where will be built, the actual connection of these inlets will be as Y connection to the pipes and manhole. I know that can be done by using Tap to the pipe but the way it is connected to the manholes is just for the purpose of modeling.
This could be at the root of the problem though. If possible, try modeling the configuration as it exists in the real system.
Mohamad Azzam said:I hope you can find the solution for this defect in order to finalize the model urgently,
Since it appears that this is preventing you have continuing your work progress, I have escalated this (reference # 788333) and will keep you updated.
Mohamad Azzam said:the run of each scenario takes a lot of time, is that can be the investigation and improved by the Bentley developer to enhance the run time?
Was this with the Implicit or the Explicit (SWMM) solver? Typically slower performance is to be expected for a larger, more complex model. Increasing the Routing Step and/or Reporting Timestep can be an acceptable compromise in some cases, though if there are quick changes or sharp hydrograph peaks in your model, a relatively small Routing Step may be necessary. If the model is struggling to converge, this can also cause a slowdown in performance (if each timestep is taking many trials to converge). In these cases, you'll want to address the root cause, which could be data entry issues (for example the large number of conduits connecting to a single manhole) and/or potentially the need for tweaking of the advanced calculation options. With the SWMM solver, you can also adjust the number of CPU threads (for multi-core CPUs) used with the calculation option "Number of Threads". Test the performance of "4", "8" or "automatic. See more guidance in the following article:
Improving calculation performance (run time) for a SewerGEMS or SewerCAD model
Mohamad Azzam said:seems the program does not take the graphic card benefit during the run. I hope this can be considered.
Currently the solvers in SewerGEMS utilize CPU cores, not the GPU / CUDA. For the SWMM solver, this would be something that the US EPA would need to implement in a new SWMM version, which SewerGEMS could then incorporate. Regardless, I have notified our Development team of your request/feedback via Enhancement # 789423.
I wanted to let you know that our developers are still working on this. In the meantime, I possible workaround is ti adjust the "Minimum Surface Area" calculation option. I was able to get the model to complete (with low non-convergence and continuity error) if I set it to 0.1 ha. I am checking with our developers if this is a valid solution but I wanted to share it since I know you need a solution to proceed with the project.
As sent earlier the screenshot below it does show the problem, we found the solution and the error source which is the whole problem is relating to the Catch basin where some inlets have no gutters and they are on grade, I convert them to "In Sag" and the program run again.
it seems this error should be as a red flag in the notification error or might be reported in another way, I hope that helps in your final conclusion and decision.
Hi Mohamad, yes, our developers also discovered that the problem is due to on-grade inlets with no downstream gutter and are working on a solution. There is already a user notification for this case, but it is just an information message, which you can see when using Validate. We will seek to display a better message and possibly stop the calculation (red instead of blue notification) so that the user must fix the problem before the run will complete.
There are 22 locations in your model where this situation occurs (use Validate to see), where one or more gutter is sloping down towards the inlet with no bypass gutter. This suggests that they likely should be changed to be in-sag. This would be the solution to your situation (or define a downstream gutter if they truly are on-grade), which I have now documented in the following article in our Wiki:
"Access violation in module execRouting" when computing with SWMM solver
Answer Verified By: Mohamad Azzam
Hello ,
This is engineer Ahmed I'm working with engineer Mohamad Azzam in the same model ,
After updating the catch basins the model is running and no failure occurs ,however, after minor updates in the conduits locations for the catch basins , the model is running for 2 hrs. without any issues but after 2 hrs. durations the model give the same message that the swmm engine failed.
For example 2 hrs. duration will keep the model running and the results will be clear but 4 hours durations the model will fail.
As attached there're no validation errors .
The model shared via Bentley secured link.
Ahmed
Hi Ahmed,
Thanks for sharing the new model, I can reproduce and will work with our developers to find a fix. It could still be related to issues with inlet configurations, which happen to not have a user notification.
Ahmed and Mohamad,
The problem now is with the cases I had previously pointed out where an extremely large number of pipes connect to a single manhole. There is currently a limit of 15 conduits that can connect to a manhole if that manhole is using the HEC-22 structure loss method. For example MH-1-7 has 18 inflow pipes which exceeds this, causing the crash in cases where more than 15 of the pipes need to be calculated.
We can investigate increasing this limit but it may take time to ensure that there are no adverse effects. In the meantime, a more realistic solution would be to modify these locations to more closely match the real system conditions. Another possible workaround would be to use another structure loss method at such manholes, such as Absolute.