[CONNECT U12] Monitor EC data change best practice?

Hi,

this my question is more about best practice, not specifically targeted to C++ or C#. C# is preferred in my current project, but when the solution will be available in C++ only, no problem.

Situation: There is an element with custom EC data attached. The data are available in Properties (Element Information) dialog and can be modified by a user. Such change is saved to persistent storage (DGN V8 file) by MicroStation automatically.

Question: Is it possible and what is recommended approch to monitor such change to be able to react on the change (e.g. to modify the element size/shape/etc.)?

I have not done any testing or research yet, but I can imagine that XAttribute changed event is fired when EC data are changed, but it seems to me a bit complicated (go through the whole path from XAttribute, test whether it's EC data or not and to what element it belongs). Maybe there is some other solution available, e.g. to monitor EC data particularly or to hook system responsible for rendering EC data in GUI?

With regards,

  Jan

Parents
  • Hi Jan,

    We have internal native way to monitor changes to XAttributes that store ECInstances.  Those APIs are not exposed yet. It may significantly impact performance of Microstation especially while processing bulk elements containing ECInstances. 

    Still, can you please file an idea. I will connect with you for more inputs on that idea.

    Btw, you can explore Bentley.ECObjects.UI.ECPropertyPane.OnPropertyValueChanging. This delegate fires up when you try to change any property from property pane. It gives you beforevalue and after value. You can reject afterValue by returning false. 


    This is a test

  • Hi Mangesh,

    Performance: this sounds very foreboding. Performance is pretty critical to adoption.

    Would 'monitoring' these changes be quicker in an i.dgn or i.model (which I believe is based on SQLite)?

    Thanks

  • Hi Dominic,

    The non exposed API I have today, if I expose as is, you will start getting callback for every EC xattribute change. So, lets say, you are interested just in your custom schema. Then you need to write a code to filter those attribute that matches to your schema. And, Microstation needs to wait till you complete your processing. Therefore I am saying it may cause performance issue. 

    May be I can make it restrictive like I may fire those events for particular schema or class. Or I may not fire it on multi element selection. Or I may give you options to make it restrictive. I am not 100% sure at this point of time. I need to write good new tests before exposing these APIs.

    Thanks,

    Mangesh


    This is a test

  • Hi Mangesh,

    Thanks for the response.

    It sounds like SQLite would not help, which is surprising since I would have thought that relational databases would have to be able to deal with 'monitoring' quickly and efficiently.

    Callbacks: we sometimes see 'maximum callbacks... ' exceptions. I guess that this is the indiscriminate unmanaged callback avalanche you are referring to.

    Waiting: I hear that AutoCAD devs complain about their apps being hard to debug because they can not tell when AutoCAD is finished processing and is back in 'quiescent' mode. I think that another problem is there may be many subscribers.

    Not sure how the sequencing of the callbacks between multiple subscribers will be managed?

    Marshalling: I think that Revit also provides 'marshalling' to help speed things up. I suppose that there is an equivalent in the database world where queries are re-jigged at low level to maximize caching etc.

    Apparently Revit also checks the propagation of the callbacks before it executes them to see if there are problems (circular dependencies etc). 

    It sounds like you have a lot of tests to do...

  • Hi Dominic,

    Would 'monitoring' these changes be quicker in an i.dgn or i.model (which I believe is based on SQLite)?

    Because i-model is designed to be read-only (as implemented in MicroStation), there is probably no reason to think about monitoring changes. But technically:

    • .i.dgn is just DGN V8, so there is probably no difference between "normal DGN" and .i.dgn.
    • .imodel is SQLite database, where complete data are stored as EC data, so also standard DGN structures and elements are converted to EC format. It means "to monitor EC change" is converted to "monitor any change and save to database".

    Regards,

      Jan

Reply Children
  • Because i-model is designed to be read-only (as implemented in MicroStation), there is probably no reason to think about monitoring changes.

    I don't think this is correct at all. If you look at most BIM workflows. The format of choice is IFC, which is 'read only'. But, constantly being edited. Do a web search and the role of GUID's also pop up a lot, highlighting that this is an iterative process. See also Bentley's own I-model Transformer tool. I think that read only is a voluntary restriction, not an inherent one.

    •.i.dgn is just DGN V8, so there is probably no difference between "normal DGN" and .i.dgn

    Also, i.dgn's  are the only way in many cases to access the data generated by the vertical. So, there is a big difference between a dgn and i.dgn which has all the vertical's information exposed.

    It is unrealistic to think that this information would not need to be monitored.

    .imodel is SQLite database, where complete data are stored as EC data,

    OK... but what about my original question: it is faster/more efficient to use this format to monitor and react to changes?

  • i.dgn's  are the only way in many cases to access the data generated by the vertical

    That was the original purpose of the i-Model: to eliminate app-dependent DGN structures that could not be displayed in vanilla MicroStation or other PowerPlatform viewer.

    there is a big difference between a dgn and i.dgn which has all the vertical's information exposed

    Whan an app user publishes (Bentley terminology) a DGN to an i-Model, the internal app-specific graphic structures are removed on being simplified to primitive DGN graphics.  Business information (i.e. EC Schema data) is retained.

    Because i-model is designed to be read-only (as implemented in MicroStation)
    I don't think this is correct at all. I think that read only is a voluntary restriction, not an inherent one

    It is correct.  That is one raison d'être of the i-Model concept.  i-Model is read-only by design.

    Do a web search and the role of GUID's also pop up

    A GUID is nothing more than a piece of data attached to an object.  One hopes and expects that a GUID persists, which in a read-only situation of course it does.  If someone copies and edits, say, an IFC or i-Model file, then one expects and hopes that the editor will persist the GUID of modified objects correctly.

    Editing, of course, introduces a risk.  What if the editor does not persist GUIDs adequately?  For example, let's say I copy an i.dgn to a writable .dgn file.  Then I edit that copy.  If my editing splits a widget into two parts, to which part should the original GUID be attached?  Or should something else happen, such as creating two new GUIDs?   How do we correlate the two new half-widgets with the original whole widget?  Those questions return us to the origin of this thread: Jan's question about monitoring EC Data changes.  Only if our code can monitor EC Data changes successfully can we think about devising a policy for propagating GUIDs when using MicroStation as an editor.

     
    Regards, Jon Summers
    LA Solutions

  • It is correct.  That is one raison d'être of the i-Model concept.  i-Model is read-only by design.

    This is becoming silly. So, how do explain i-model Transformer? In any case, what the religious meaning  / intention of i.dgn's is secondary. What the users do with it will count for more. 

    Editing, of course, introduces a risk.

    Exactly. That's why there needs to be some tools to help persist and manage / resolve changes... GUID management has always been a hot topic.

    Sticking your head in the sand does not help. The need is there. 

  • Hi Dominic,

    I don't think this is correct at all. If you look at most BIM workflows

    I am not sure whether your reaction is that my formulation (i-model is read only) is wrong or you disagree with the concept or read only repository in a context of e.g. BIM workflow.

    Technically, i-model in Power products is read only, regardless .i.dgn or .imodel is used. End of story.

    I do not know how i-model is used in ConceptStation product, my assumption (but I have nearly zero knowledge here) is they used i-model in different way as standard repository.

    But e.g. iModelJS, next generation of i-model, is also strictly read only format, and modifications are stored as "change transactions", so from users' perspective it looks like read/write format, but it's not. This approach is not new and is used by many products, probably all code repository and versioning systems (where Git is I guess direct inspiration for iModelJS) work this way, also Adob Lightroom stores all picture modifications as a chain of time ordered transcations, but original data are never modified.

    So, there is a big difference between a dgn and i.dgn which has all the vertical's information exposed.

    Technically there is nearly no difference between .dgn and .i.dgn, the format is the same, but as Jon wrote, data is converted into different structures when published to i-model. It's primarily about data optimization, but not primarily about format change: e.g. SmartSolid is converted from complicated structure storead as cell with huge amount of custom data to standardized representation, vertical applications convert data from proprietary UserAttributes and XAttributes structures to EC schemas and EC classes. They can do the same when working with original DGN, but from historical and subtle technical reasons they do not in current versions.

    it is faster/more efficient to use this format to monitor and react to changes?

    In context of MicroStation the question does not make sense, because i-model is read only, so no change can be propagated to this format after it's created.

    Regards,

      Jan

    P.S. It seems we have gone far from my original question....

  • So, how do explain i-model Transformer?

    As far asI know, i-model Transformer is designed similarly to ETL (Extract Transform Load) tools: No change is done at original data, but data are read, transformed (converted, merged with another sources etc.) and saved to new "data version" with own provenance information.

    That's why there needs to be some tools to help persist and manage / resolve changes...

    I think that's one from main topics behind iModelJS.

    Sticking your head in the sand does not help.

    I do not understand this comment and it seems to be offensive a bit.

    Because we are in programming forum, as developers we can discuss what is possible to do and what is possible to implement using existing APIs and what features are allowed by used formats. Nothing more, nothing less. We depend, or better to say we choose to depend ;-), on what Bentley decides that is the best to fulfill their business strategy.

    Of course there are alterantive solutions (every new product tells it solves all existing questions and issues ;-) and ideas how existing need can be solved, but in my opinion it's beyond scope of this forum. Fortunately anybody can go and to investigate own technology, but because after so many years there is still no technology that beats other and conquers the world, it's probably extremely complicated at the same time to design it, to implement it in acceptable cost and be able to sell it.

    Regards,

      Jan

  • I am not sure whether your reaction is that my formulation (i-model is read only) is wrong or you disagree with the concept or read only repository in a context of e.g. BIM workflow

    Yes to both. The info provided via i.dgn or I.model may start out being read only, but this will often (almost certainly given long enough) be written to as well when it is consumed by another. Taking the position that it is read only is neither here nor there. It is not the End of the Story as you say, rather the beginning. An i.dgn provided by a designer will be amended by the cost estimator, when he adds the cost rates. An i.dgn with terrain info will be amended by others like a geotechnical or health and safety engineers when they consume the info. This will need happen interactively and tracked.

    ConceptStation product

    Exactly. My understanding is that they are using the SQLite format. So, write as well as read. I think that Navigator Mobile also must have some write capability as it is being used to take notes on site.

    iModelJS, next generation of i-model,

    Again, I think that you are mistaken. OpenPlant is transitioning to Imodel.Hub.

    Technically there is nearly no difference between .dgn and .i.dgn

    I think that we are talking past each other. The difference as far as the user is concerned is that the 'business' and other information that is provided by the vertical is converted to Items and can be read without the need for the originating app. The geometry is important but secondary in this case. ADSK's Object Enablers also provide something similar.

    One problem that is already manifest when everything is 'read only' is the cack-handed way Bentley handles objects that have been generated by a vertical, when they are are referenced or opened in another app. They can not be manipulated in any way (Element Handler(?) blocks the Mstn Move tool for example). So, a component generated by a vertical can not be moved into its correct position unless the originating app is loaded, turning it into a 'zombie'.

    In context of MicroStation the question does not make sense, because i-model is read only, so no change can be propagated to this format after it's created.

    This sentence does not make any sense as your premise is erroneous. SQLite is a relational database so is designed to take writes... just like .dgn.

    And yes, leaving aside the question of whether writes should be allowed, there are many read-only applications.

  • .i.dgn-based iModel is read-only. sqlite-based iModel is all about change, in pretty much exactly the way Jan describes, therefore not read-only. But MicroStation does not allow directly editing it. iModel.js can be used to create apps which treat the iModel as read-only or allow editing it. Navigator Mobile does not write to the iModel; its saved views, annotations, etc are saved to a separate location shared amongst users of a project. ConceptStation does write to the iModel.

    None of this discussion is helping to answer the original question.

  • So, SQLite-based I.models allow change. As does .iModel.js.

    Question for Paul regarding change monitoring:would this be easier / faster using SQLite based i.models?

  • regarding change monitoring:would this be easier / faster using SQLite based i.models?

    Please post a new question.  We've deviated a long way from Jan's original question.

     
    Regards, Jon Summers
    LA Solutions