[CONNECT C#] Faster method to extract all element Item Type Property values?

Hi all,

I am trying to cycle through the elements in a dgn model and extract all Item Type property values for them but am finding that the EC IDgnInstance GetPropertyValue method is quite slow (takes 7 seconds per element that has approx 20 values). I need to do this for 1000s of elements so this method will take hours to run.

Does anyone have any ideas to get Item Type property values extracted to a List faster? I am using the latest version of Microstation (U17.1).

Note that it looks to bog down for string values that were generated from Expressions.

using Bentley.DgnPlatformNET;
using Bentley.DgnPlatformNET.DgnEC;
using Bentley.DgnPlatformNET.Elements;
using Bentley.ECObjects;
using Bentley.ECObjects.Instance;


CustomItemHost customItemHost = new CustomItemHost(element, false);
IEnumerator<IDgnECInstance> customItemList = customItemHost.CustomItems.GetEnumerator();

while (customItemList.MoveNext())
{
    IDgnECInstance customItem = customItemList.Current;
    ItemType itemType = null;

    foreach (ItemTypeLibrary itemTypeLibrary in ItemTypeLibraries)
    {
        if (itemTypeLibrary.HasInstancesInFile && itemTypeLibrary.ItemTypeCount > 0)
        {
            itemType = itemTypeLibrary.GetItemTypeByName(customItem.ClassDefinition.Name);

            if (itemType = null)
            {
                break;
            }

            CustomPropertyContainer itemTypeCont = itemType;

            foreach (CustomProperty customProperty1 in itemTypeCont)
            {
                if (customProperty1 != null)
                {
                    string propName = customProperty1.InternalName;
                    object propValOut = null;
                    IECPropertyValue propVal = null;
                    propVal = customItem.GetPropertyValue(propName);
                    int type = (int)customProperty1.Type;
                    
                    
                    if (type == 0)
                    {
                        propValOut = Convert.ToBoolean(propVal.IntValue);
                    }
                    else if (type == 2)
                    {
                        propValOut = propVal.DoubleValue;
                    }
                    else if (type == 3)
                    {
                        propValOut = propVal.IntValue;
                    }
                    else if (type == 4)
                    {
                        DPoint3d dpoint3d = DPoint3d.FromXY(0.0, 0.0);
                        dpoint3d = (DPoint3d)propVal.NativeValue;
                        propValOut = (string)(dpoint3d.X + "," + dpoint3d.Y);
                    }
                    else if (type == 5)
                    {
                        propValOut = propVal.StringValue;
                    }
                    else
                    {
                        propValOut = propVal.NativeValue;
                    }
                }
            }
        }
    }

Regards,

Mark

  • Maybe to ensure Expression are written in efficient way is bettersolution. Although I alwyas tell and write that Expressions represents overhed and it is not "an aid for everything and anything", I do not recall realy issue with Expressions. And what I remember from a presentation about Item Types new features in ORD 10.11, it was mentioned that even hundreds thousands of attache items, with expressions defined, does not cause any substatnial slow down.

    Or maybe it is the issue of 10.09 / 10.10? Did you try it with 10.11?

    Funny you say that, the aim of this code is to resolve an issue with IFC export of ORD 10.11. In both 10.10 and 10.11 it takes us hours (3-4) to write an IFC that includes Item Type data, and when no Item Type data is included it takes 5-10 minutes. It is looking like this issue is the culprit. Unfortunately we can't simplify our Item Types any more to get the automation we are looking for, but need to find a faster way to extract data out of our models. Even exporting Item Type data using the Excel Export Instances tool takes a very long time - something our users have started to do to reimport "static" data just for export.

    There have been performance gains in ORD 10.11 in this regard, but nothing substantial enough to save hours in this regard unfortunately.

    Regards,

    Mark


    OpenRoads Designer 2022 R3 (10.12)  |  Microstation 2023.1  |  ProjectWise CE 3.4

  • I take it there is no "Read-only" method to evaluate the Item Type values?

    I think there is no such method.

    I am not sure (but maybe can explain this aspect), but maybe when EC Expression is defined, and "Use last valid value" is not used, the value is not even stored and is alwyas recalculated?

    Maybe to ensure Expression are written in efficient way is bettersolution. Although I alwyas tell and write that Expressions represents overhed and it is not "an aid for everything and anything", I do not recall realy issue with Expressions. And what I remember from a presentation about Item Types new features in ORD 10.11, it was mentioned that even hundreds thousands of attache items, with expressions defined, does not cause any substatnial slow down.

    Or maybe it is the issue of 10.09 / 10.10? Did you try it with 10.11?

    Would VBA interop methods like Property Handler behave the same way?

    Yes. As far as I know, at background PropertyHandler use code close to Properties dialog, so I assume EC Expressions are evaluated too.

    With regards,

      Jan

  • Thanks Jan,

    That is very helpful. I take it there is no "Read-only" method to evaluate the Item Type values? Would VBA interop methods like Property Handler behave the same way?

    Regards,

    Mark


    OpenRoads Designer 2022 R3 (10.12)  |  Microstation 2023.1  |  ProjectWise CE 3.4

  • Hi Mark,

    it seems like the Item Type expressions are being recalculated as the item types are being read

    Yes, it is how EC Expressions are designed: They are evaluated every time the value "is asked" (like an element is selected and displayed in Properties dialog or the data is read by code).

    The ones which have if statements involving numerical calcs run slower than the rest and some based on Expressions with a State value of "NotYetCalculated" take even longer.

    I do not recall any sych benchmarks. EC Expressions are "just string", that is evaluated and interpreted when used. So there is alwas some overhead, because string operations are expensive, but it's typically not an issue. So I assume the real source of the slowness are expressions itself or how the symbol provider is implemented.

    I think there are not best practices available how to write efficient Expression (and what to don't do), it's more "knowledge based" process.

    To evaluate what Expression slows down the process, I can imagine to add a filter to code, so only not calculated properties are processed. If the different is big, it makes sense to analyze expressions step by step.

    I'll be trying Manesh's suggestion to see if it helps.

    I think Mangesh's approach is nearly eqaul to my code. The main difference is that I do not use EC Query to access (potentiall all) EC data, because you mentioned you are interested in Item Types only.

    When you have ECInstance and you know ECClass, how to access propertis values are always the same.

    With regards,

      Jan

  • Thanks Jan, that looks to be much more cleaner than what i had and works a bit better.

    it is a bit faster, but still much more excessive than expected - it seems like the Item Type expressions are being recalculated as the item types are being read into my code. The ones which have if statements involving numerical calcs run slower than the rest and some based on Expressions with a State value of "NotYetCalculated" take even longer.

    I am only really interested in extracting the Item Type text and numerical values per element for an export addin i am putting together. I'll be trying Manesh's suggestion to see if it helps. There looks to be something strange happening with these Expressions...

    Regards,

    Mark


    OpenRoads Designer 2022 R3 (10.12)  |  Microstation 2023.1  |  ProjectWise CE 3.4

  • Hi Mark,

    Your code snippet will be slower. You are iterating each element, then for each element, you are doing itemTypeLibrary.HasInstancesInFile. This method will query the whole dgn to check if all library classes have instances. I think you don't need the following loop:

    foreach (ItemTypeLibrary itemTypeLibrary in ItemTypeLibraries)
    {
    if (itemTypeLibrary.HasInstancesInFile && itemTypeLibrary.ItemTypeCount > 0)
    {

    And, with Direct EC APIs you can avoid iterating all elements:

    1. Create ECQuery for all classes.  ECQuery readGadget = new ECQuery(<list of all classes>);readGadget.SelectClause.SelectAllProperties = true;

    2. Create scope: 

    FindInstancesScope scope = FindInstancesScope.CreateScope(activeDgnModel.GetDgnFile(), new FindInstancesScopeOption(DgnECHostType.Element));

    3. Execute:

    DgnECInstanceCollection instanceCollection = dgnECManager.FindInstances(scope, readGadget);

    Please try SDK sample: MstnExamples\DgnEC\DgnECCrudExample\ManagedExample.

    Thanks,

    Mangesh


    This is a test

  • Hi Mark,

    I do not know a structure of your files, but code similar to this one should work. I had no time to test it in detail, but it lists "something" when used with simple test DGN.

    DgnModel dgnModel = Session.Instance.GetActiveDgnModel();
    ModelElementsCollection elements = dgnModel.GetGraphicElements();
    
    foreach (Element element in elements)
    {
        CustomItemHost host = new CustomItemHost(element, false);
    
        IList<IDgnECInstance> items = host.CustomItems;
    
        foreach (IDgnECInstance item in items)
        {
            IEnumerable<IECProperty> properties = item.ClassDefinition.Properties(false);
    
            StringBuilder sb = new StringBuilder();
    
            foreach (IECProperty property in properties)
            {
                IECPropertyValue value = item.GetPropertyValue(property.Name);
    
                if (value is null) { continue; }
    
                sb.AppendLine($"Property '{property.DisplayLabel}' = {value.StringValue} ");
            }
    
            string msg = $"Element id {element.ElementId} / Item {item.ClassDefinition.DisplayLabel}";
            string details = sb.ToString();
            NotifyMessageDetails message = new NotifyMessageDetails(OutputMessagePriority.Information, msg, details, NotifyTextAttributes.None, OutputMessageAlert.None);
            NotificationManager.OutputMessage(message);
        }
    }

    There is a space for improvements, like to use value formatters, plus this snippet does not support arrays.

    With regards,

      Jan

  • Hi Mark,

    takes 7 seconds per element that has approx 20 values

    7 seconds sounds absurdly slow, I would expect something like 7 ms.

    I am trying to cycle through the elements in a dgn model and extract all Item Type property values

    I do not work with Item Types API, as I typically use general EC data API for my own data structures, so my knowledge is limited a bit, but your code looks weird. I must think about it.

    Are you interested in all EC data (both intrinsic and extrinsic), or specifically in Items, created by a used as Item Types?

    Note that it looks to bog down for string values that were generated from Expressions.

    Despite of it may be true, I don't think so: In OpenRoads Designer presentation it was mentioned that expressions cause some slow-down, but it would be minimal.

    With regards,

      Jan