ODC polling of a database has a physical limitation of the number of indicator readings that can be processed during a reasonable recurrence pattern. To avoid this limitation, a user attempted to dataload indicator readings instead within the same recurrence timeframe. The user found that although performance was improved using the indicator reading dataloader, there is still a physical limitation in indicator reading dataload processing of approx. 20K readings per day in their environment. This physical limitation resulted in the indicator dataload process running for longer than the allotted recurrence period of 24 hours. This caused the Indicator Reading dataload process to restart prior to marking the Indicator Reading dataload Process Job as complete also resulting in the Indicator Reading dataload records not be marked as "Processed". This led to a duplicate Indicator Reading Process Batch being created, which included all of the previously processed indicator readings which did not have their status updated to "Processed" or "Errored".
Bentley APM Development has launched a new project to research a new approach to ODC Indicator Reading Processing and Dataloading Indicator Readings processing to improve indicator processing performance time and reduce physical limitations. A permanent fix is being worked on for future release.
In the immediate term, APM hotfix 7.10.0.212 has several fixes which significantly improves Indicator Reading processing performance. Additionally there are several Workarounds suggested to be followed when trying to improve the likelihood of failed dataload or ODC Poll:
1. Reducing the size of the Indicator Reading dataload batches to allow completion of the processing prior to the next scheduled recurrence.
2. Reducing the number of Indicators in a Polling schedule for ODC polling indicators and create additional Polling Schedules.
Please issue a Service Request should you run into these problems described here.