[CONNECT C#] Faster method to extract all element Item Type Property values?

Hi all,

I am trying to cycle through the elements in a dgn model and extract all Item Type property values for them but am finding that the EC IDgnInstance GetPropertyValue method is quite slow (takes 7 seconds per element that has approx 20 values). I need to do this for 1000s of elements so this method will take hours to run.

Does anyone have any ideas to get Item Type property values extracted to a List faster? I am using the latest version of Microstation (U17.1).

Note that it looks to bog down for string values that were generated from Expressions.

using Bentley.DgnPlatformNET;
using Bentley.DgnPlatformNET.DgnEC;
using Bentley.DgnPlatformNET.Elements;
using Bentley.ECObjects;
using Bentley.ECObjects.Instance;


CustomItemHost customItemHost = new CustomItemHost(element, false);
IEnumerator<IDgnECInstance> customItemList = customItemHost.CustomItems.GetEnumerator();

while (customItemList.MoveNext())
{
    IDgnECInstance customItem = customItemList.Current;
    ItemType itemType = null;

    foreach (ItemTypeLibrary itemTypeLibrary in ItemTypeLibraries)
    {
        if (itemTypeLibrary.HasInstancesInFile && itemTypeLibrary.ItemTypeCount > 0)
        {
            itemType = itemTypeLibrary.GetItemTypeByName(customItem.ClassDefinition.Name);

            if (itemType = null)
            {
                break;
            }

            CustomPropertyContainer itemTypeCont = itemType;

            foreach (CustomProperty customProperty1 in itemTypeCont)
            {
                if (customProperty1 != null)
                {
                    string propName = customProperty1.InternalName;
                    object propValOut = null;
                    IECPropertyValue propVal = null;
                    propVal = customItem.GetPropertyValue(propName);
                    int type = (int)customProperty1.Type;
                    
                    
                    if (type == 0)
                    {
                        propValOut = Convert.ToBoolean(propVal.IntValue);
                    }
                    else if (type == 2)
                    {
                        propValOut = propVal.DoubleValue;
                    }
                    else if (type == 3)
                    {
                        propValOut = propVal.IntValue;
                    }
                    else if (type == 4)
                    {
                        DPoint3d dpoint3d = DPoint3d.FromXY(0.0, 0.0);
                        dpoint3d = (DPoint3d)propVal.NativeValue;
                        propValOut = (string)(dpoint3d.X + "," + dpoint3d.Y);
                    }
                    else if (type == 5)
                    {
                        propValOut = propVal.StringValue;
                    }
                    else
                    {
                        propValOut = propVal.NativeValue;
                    }
                }
            }
        }
    }

Regards,

Mark

  • but there is one creative GetRelatedInstance expression that may be exacerbating the whole thing.

    Personally I do not think the usage of related instance can cause any substantial slow down: the relation between EC classes is a part of native EC data, so to read the relation definition to obtain access to target instance should be equally fast as to read normal EC data. Of course it depends how GetRelatedInstance method is implemented, but at least, it is about access inside DGN.

    Noting that the majority of values are Excel lookup string values

    I have no experience with lookup tables in Excel, when used from Item Types, but I can imagine it can be slow, because external file must be opened and read.

    It would be great if somebody from Bentley is able to share best practices and what to check, when external tables are used so masivelly.

    With regards,

      Jan

  • Interestingly, using Interop Property Handler to read the Property values is exponentially faster. I'm thinking this may not be re-evauating the same way as EC direct methods. A little more messy than i was after, but SEEMS to work well...

     List<PropertySetIfc> properties = new List<PropertySetIfc>();
                                CustomItemHost host = new CustomItemHost(meshHeader, false);
                                IList<IDgnECInstance> items = host.CustomItems;
    
                                BIM.Element eleInter = ElementOperation.ConvertToInteropEle(meshHeader);
                                BIM.PropertyHandler oProps = BMI.Utilities.ComApp.CreatePropertyHandler(eleInter);
    
                                foreach (IDgnECInstance item in items)
                                {
                                    PropertySetIfc ps = new PropertySetIfc(item.ClassDefinition.Name, "");
                                    IEnumerable<IECProperty> itemProperties = item.ClassDefinition.Properties(false);
    
                                    foreach (IECProperty itemProperty in itemProperties)
                                    {
                                        string itemPropName = itemProperty.Name;
                                        MessageCenter.Instance.ShowMessage(MessageType.TempRight, "Scanning active model...Scanning Mesh " + (csCount + 1).ToString() + " of " + meshCount.ToString() + " > reading Item Type: " + itemPropName, "", MessageAlert.None);
                                        string unitName = "";
                                        object propValOut = null;
    
                                        try
                                        {
    
                                            oProps.GetAccessStrings();
                                            if (oProps.SelectByAccessString(itemPropName))
                                            {
                                                object propVal = oProps.GetDisplayString();
                                                propValOut = propVal;
    

    Regards,

    Mark


    OpenRoads Designer 2022 R3 (10.12)  |  Microstation 2023.1  |  ProjectWise CE 3.4

  • Interestingly, using Interop Property Handler to read the Property values is exponentially faster.

    Wow, it is something I did not expect Open mouth

    Is there any reason for the behavior Mark described?

    Regards,

      Jan

  • CpropertyHandler is a COM wrapper over native ECInstance. When you call GetDisplayString or GetValue, we are directly returning a native value. It doesn't involve managed-to-native marshalling and no intermediate callbacks. Moreover, when you try Civil elements at managed layer, I am doubting that they may have many other handlers or callbacks which might make things slower. 

    Mark, please raise this to Civil.


    This is a test

  • Thanks guys, will do

    Regards,

    Mark


    OpenRoads Designer 2022 R3 (10.12)  |  Microstation 2023.1  |  ProjectWise CE 3.4

  • Hi all,

    We think we are closer to tracking down the issue. It seems that calculated values using IIF statements is slowing down the extraction of these properties substantially. Why would this be the case?

     

    Attached are the expressions we are using. Note that this was made for Civil GetRelatedInstance expressions, hence all references to corridors, etc. but we have confirmed that the GetRelatedInstance expression has minimal effects on performance.

    TfNSW_DE_Data_Arcadis.xlsx

    Regards,

    Mark


    OpenRoads Designer 2022 R3 (10.12)  |  Microstation 2023.1  |  ProjectWise CE 3.4

  • Hi Mark,

    Why would this be the case?

    I think it's hard to guess without knowing how individual functions are implemented.

    but we have confirmed that the GetRelatedInstance expression has minimal effects on performance.

    To reference related instance is efficient, because it is already existing relationship, so (I guess) the only difference is that one more extra EC instance must be read.

    It seems that calculated values using IIF statements is slowing down the extraction of these properties substantially.

    I do not think IIF itself is slow (although as I wrote, nobody know how exactly it is implemented ;-). But the used conditions, based on string operations, can be expensive (generally, in software development, string operations are one from the most expensive, because when not used carefully, require plenty of memory allocations and data iteration and comparing).

    I think, when complete data is available, it would be possible (at least for NET code) to do some benchmarks, to see what code is (A) called the most often and (B) what method in code consumes the most time.

    With regards,

      Jan

  • Hi Jan,

    Looking at this further, it looks like every time any IIF statement is trying to access an Item Type that contains an expression, that expression is reevaluated. Based on the nature of EC expressions that behaviour makes sense, however is extremely expensive performance wise for any practical use of a value derived from an Expression elsewhere.

     

    Having the ability to access these values "read-only" surely would aid in general performance increases for operations on Expressions globally surely? I'm seeing that reevaluation looks to be also happening with data from read-only sources like external references.

     

    Regards,

    Mark


    OpenRoads Designer 2022 R3 (10.12)  |  Microstation 2023.1  |  ProjectWise CE 3.4