Problem

Server Manager reports that all Servers are Stale.

This condition can be artificially reproduced by interrupting all communication to the database for more than 2 minutes.  However, in this controlled test, the APMServer instances keep trying the database, and when it is  brought back on line, the APMServers recover and reported status reverts to running, or whatever is normal for each of the APMServer instances.

In the case of this SR, continued refreshes of the Server Manager Home page continues to show that the APMServers did not recover on their own, they remained in Stale status

Solution

To troubleshoot this situation, Support would require the following information to be collected, prior to restarting of the services:

In Server Manager

1. Take a print screen to capture all data points on the Home screen (interested in Las Report and Up Since columns, among others)
2. Hit refresh to make sure it is not auto-recovering
3. Go to Client Requests Page – Download full request logs
4. Go to Logs Page – Download full log
5. Go to Monitor page – If information comes up, print screen it

On each of the Machines running Application Server processes:

1. Launch task Manager and print screen to show ivaraserver processes that are still up
2. Take 2 memory dmps of each ivaraserver.exe process that is running
3. Zip the log folder (default is C:\ProgramData\Bentley\IvaraEXP\Logs)
4. Restart the services

Zip all artifacts collected request a upload link when submitting your Service Request.

See Also

Reporting a Server crash or hang to APM Support: https://communities.bentley.com/products/assetwise/asset_performance_management/w/wiki/28759/reporting-a-server-crash-or-hang-situation-to-apm-support