- Import multiple folders of Shapefiles by using GDI key-ins

A benefit of using OpenCities Map is the ability to import Shapefile data into your project. For most users, this means using the Map Interoperability utility and selecting a single folder containing multiple Shapefiles. However, when multiple folders of data need to be imported, separate import nodes must be defined. While this is fine for few folders, it becomes problematic when dealing with dozens or even hundreds of folders. This tutorial will present a few ideas to help address this problem.

Let's suppose that a municipality is doing a field audit on items such as fire hydrants, catch basins, traffic signs and more. The data collector captures the location and related attributes on a street by street basis and uploads this data at the end of every day to a central server where it will be accessed by OpenCities Map users. In this example dataset, folders named Maple Blvd, Maple Drive and Maple Street all have the same named shapefiles.

As already mentioned, the Map Interoperability dialog only allows a single directory to be imported at a time. And while this will work in a pinch, it's certainly not efficient for bulk importing of data that's spread over many folders. To deal with bulk importing of data, let's take a look at using the GDI key-ins which are part of core OpenCities Map.

The GDI key-ins:

The GDI set of key-ins allow you to perform import and export operations and, for the purposes of this article, are described as follows:

gdi open name=[STORAGE NAME] file=[FILE or FOLDER] - Opens the specified folder or file and associates it with a storage name of your choosing. If the file or folder name has spaces, enclose in double quotes. In our sample, this key-ins would be:

gdi open name=blvd file="c:/Bentleyville/maple blvd"
gdi open name=drive file="c:/Bentleyville/maple drive"
gdi open name=street file="c:/Bentleyville/maple street"

The gdi open command can also be used to open an Oracle source - please refer to OpenCities Map help for more information.

gdi import storage=[STORAGE NAME] feature=[FEATURE] options - Once a storage name has been defined, the data is imported via the gdi import command. This command specifies the storage name as well as the feature name which is the name of the Shapefile for this article. It can also be used to specify a feature that's been defined in an associated XML project. The gdi import key-in has several command line options that drive the properties, level, colour, weight, class, transparency, fill, cell, cell library, scale and so forth. Please refer to OpenCities Map help for a full list of options and their usage.

For purposes of this article, the key-in will be used to place a cell for the each point. An example is:

gdi import storage=blvd feature=CatchBasin cell=CatchBasin
gdi import storage=blvd feature=FireHyd cell=FireHyd
gdi import storage=blvd feature=SurveyMon cell=SurveyMon

gdi close name=[STORAGE NAME] - This is the last key-in to be executed and it simply closes the storage that was defined and imported above. An example key-in would be

gdi close name=blvd

Bringing it all together:

Using the GDI key-ins is well suited for running a script to do the bulk import. A typical script might look like the following:

m,stBulk Import Started.
RC=$(_USTN_PROJECTDATA)cell\Bentleyville.cel
AS=1
gdi open name=blvd file="c:/Bentleyville/maple blvd"
gdi import storage=blvd feature=CatchBasin cell=CatchBasin
gdi import storage=blvd feature=FireHyd cell=FireHyd
gid import storage=blvd feature=SurveyMon cell=SurveyMon
gdi close name=blvd
gdi open name=drive file="c:/Bentleyville/maple drive"
gdi import storage=drive feature=CatchBasin cell=CatchBasin
gdi import storage=drive feature=FireHyd cell=FireHyd
gid import storage=drive feature=SurveyMon cell=SurveyMon
gdi close name=drive
gdi open name=street file="c:/Bentleyville/maple street"
gdi import storage=street feature=CatchBasin cell=CatchBasin
gdi import storage=street feature=FireHyd cell=FireHyd
gid import storage=street feature=SurveyMon cell=SurveyMon
gdi close name=street
m,stBulk SHP Import Done.

This script would be a text file run in the key-in window similar to this command: @c:/import/BulkSHP.txt

The m,stBulk Import Started and m,stBulk SHP Import Done are messages sent to the message center.

Notes and Considerations:

Refer to OpenCities Map for additional information and command line options for these key-ins. This article only touches on the subject briefly.

The symbology of the incoming data can be specified by the gdi import keyin or by feature definitions in the project schema. If you're using the project schema, the import will be somewhat slower. Also note that feature names can not contain spaces or dashes which may require remapping to accommodate the names of the Shapefiles. There is a command line option for doing that.

To create the script, get creative with DOS and perhaps EXCEL. For example, these four keyins will produce shapefiles.txt that's ready for importing. (Run them one at a time, the "for..." command does not work inside a BAT file.)

echo m,sBulk Import Started > shapefiles.txt

echo RC=$(_USTN_PROJECTDATA)cell\bentleyville.cel;AS=1 >> shapefiles.txt

for /f "delims=," %a in ('dir /s /b *.shp') do echo gdi open name=STORAGE file="%~a";gdi import storage=STORAGE feature="%~na" cell="%~na";gdi close name=STORAGE >> ShapeFiles.txt

echo m,stBulk SHP Import Done >> ShapeFiles.txt

The contents of the resulting text file are is as follows. Notice that the name STORAGE is defined for every Shapefile, then imported and immediately closed. This is a terrific solution for dealing with hundreds of folders, and while it does run a bit slower, it requires zero user interaction.