Changing group's DGNLIB to avoid conflicts when opening DWGs

You have done it correctly. By saving a DGNLIB as DWG, any and all of the features not supported by ACAD is effectively removed/lost in the process. When you open the DWG back and save it as DGNLIB, the data remains DWG compatible until you make changes in it. While it is a laborious process, you only need to do this once, so it may not be that bad after all. The only advice I have is to check you round-tripped DGNLIB to be sure the modified version while compatible with DWG now can still live up to your standard.

Don, thanks for the quick reply.

Opening the final DGNLIB, why does making any changes at all (even color) makes the DimStyle conflict (the icon turns to triangle)?
I can see why some settings would do that as it conflicts with AutoCAD, but color or line style?

Yes, it is unfortunately true. ACAD does not allow any line style to be used on any of a dimension's component such as dimension line, extension line, text or terminator. You cannot control line style or weight on these components individually. For terminators, you cannot change its color either. A way to tell what can be changed vs what cannot is perhaps by just looking at the dimension style dialog box as you have shown in your screen capture. The properties that are grayed out are the ones to pay attention to. Not all grayed out properties are due to DWG workmode restriction but they are the ones to get your focus.

Another way of finding incompatible properties is to check dimension capabilities stored in a text file \MicroStation\config\system\capability.list.  Look for CAPABILITY_DIMENSION_ prefixes.  Just make sure you do not edit/change the file as it is a data file used by MicroStation.

Yes, it is unfortunately true. ACAD does not allow any line style to be used on any of a dimension's component such as dimension line, extension line, text or terminator. You cannot control line style or weight on these components individually. For terminators, you cannot change its color either. A way to tell what can be changed vs what cannot is perhaps by just looking at the dimension style dialog box as you have shown in your screen capture. The properties that are grayed out are the ones to pay attention to. Not all grayed out properties are due to DWG workmode restriction but they are the ones to get your focus.

Another way of finding incompatible properties is to check dimension capabilities stored in a text file \MicroStation\config\system\capability.list.  Look for CAPABILITY_DIMENSION_ prefixes.  Just make sure you do not edit/change the file as it is a data file used by MicroStation.

Adjusting the settings on the final DGNLIB do you recommend being in the groups workspace or a default (empty)?
I can see benefits to either way.

The active settings gets updated/changed every time you activate a dimension style, or make style related change in a dimension tool (which effectively making the active settings out of sync of the actual style), so fixing DGNLIB is probably a better choice.