Replace dbf for csv

The current model reads in dbf and we have decided to change the dbf files to csv files. I transformed the dbf into a csv but when reading the csv im getting an error. Is there a new way to read in a csv file. Currently im trying as:

RUN PGM=NETWORK PRNFILE="{OUTDIR}\01_Zero_Volume_Delay_Network\ZVNET00B.PRN" MSG='Assign category groups based on FT'
FILEI LINKI[1] = "D:\ABQ_Model_72_pro_updated\02_APPLICATIONS\03_INITIALIZE_LOOP\00_ZERO_VOLUME_DELAY\ZVNET00A.NET"
FILEO LINKO = "{ZeroVolDelay}\ZNET00B.DBF",
include=A,B,BALINK

FILEO NETO = "{ZeroVolDelay}\ZVNET00B.NET"


FILEI LOOKUPI[1] = "{CATALOG_DIR}\01_Inputs\common\FT_CAT_Groups.csv"
;-------------------------
LOOKUP LOOKUPI=1, NAME=CAT_GROUP,
LOOKUP[1]=FT, RESULT=GROUP, ; CATEGORY GROUP ID
INTERPOLATE=F, FAIL[1]=0, FAIL[2]=0, FAIL[3]=0

;===================
;CATEGORY Category Name Categroy_Group GROUP_ID
;1 High Speed Ramps Ramps 0
;2 Urban Principal Arterials Arterials2 4
;3 Urban minor arterials Arterials2 4
;4 Urban collectors Collectors 3
;5 Urban Locals Locals 2
;6 Urban freeway frontage roads Frontage Roads 6
;7 Urban freeway links Freeway Mainline 0
;8 Urban entrance Ramps Ramps 0
;9 Urban exit Ramps Exit Ramps 1
;10 Limited Access Principal ArteriArterials1 5
;11 Rural minor collectors Collectors 3
;12 Rural Principal Arterials Arterials2 4
;13 Rural minor arterials Arterials2 4
;14 Rural major collectors Collectors 3
;15 Rural locals Locals 2
;16 Rural freeway frontage roads Frontage Roads 6
;17 Rural freeway links Freeway Mainline 0
;18 Rural entrance Ramps Ramps 0
;19 Rural exit Ramps Exit Ramps 1
;20 Rail Runner Track Rail Runner Track 0
;21 Dummy links Dummy links 0
;22 Dummy links Dummy links 0
;50 Transit-Only Links Transit-Only Links 0
;99 Centroid connectors Centroid connector 0

; collapse FTs into category-groups
GROUP = CAT_GROUP(1,FT)

;------------------
; prepare table to compute to flag oneway/twoway links in the network
;flip A and B nodes
BALINK = B*100000+A

ENDRUN

where FILEI LOOKUPI[1] = "{CATALOG_DIR}\01_Inputs\common\FT_CAT_Groups.csv" used t be FILEI LOOKUPI[1] = "{CATALOG_DIR}\01_Inputs\common\FT_CAT_Groups.dbf"

but with the new change im getting the following error


NETWORK (v.06/18/2018 [6.4.4 x64]) Wed Oct 28 10:45:30 2020

FILEI LINKI[1] = "D:\ABQ_Model_72_pro_updated\02_APPLICATIONS\03_INITIALIZE_LOOP\00_ZERO_VOLUME_DELAY\ZVNET00A.NET"
FILEO LINKO = "D:\ABQ_Model_72_pro\Scenario_tree\Model_Update\01_Zero_Volume_Delay_Network\ZNET00B.DBF",
include=A,B,BALINK

FILEO NETO = "D:\ABQ_Model_72_pro\Scenario_tree\Model_Update\01_Zero_Volume_Delay_Network\ZVNET00B.NET"


FILEI LOOKUPI[1] = "D:\ABQ_Model_72_pro_updated\01_Inputs\common\FT_CAT_Groups.csv"
;-------------------------
LOOKUP LOOKUPI=1, NAME=CAT_GROUP,
LOOKUP[1]=FT, RESULT=GROUP, ; CATEGORY GROUP ID
INTERPOLATE=F, FAIL[1]=0, FAIL[2]=0, FAIL[3]=0

;===================
;CATEGORY Category Name Categroy_Group GROUP_ID
;1 High Speed Ramps Ramps 0
;2 Urban Principal Arterials Arterials2 4
;3 Urban minor arterials Arterials2 4
;4 Urban collectors Collectors 3
;5 Urban Locals Locals 2
;6 Urban freeway frontage roads Frontage Roads 6
;7 Urban freeway links Freeway Mainline 0
;8 Urban entrance Ramps Ramps 0
;9 Urban exit Ramps Exit Ramps 1
;10 Limited Access Principal ArteriArterials1 5
;11 Rural minor collectors Collectors 3
;12 Rural Principal Arterials Arterials2 4
;13 Rural minor arterials Arterials2 4
;14 Rural major collectors Collectors 3
;15 Rural locals Locals 2
;16 Rural freeway frontage roads Frontage Roads 6
;17 Rural freeway links Freeway Mainline 0
;18 Rural entrance Ramps Ramps 0
;19 Rural exit Ramps Exit Ramps 1
;20 Rail Runner Track Rail Runner Track 0
;21 Dummy links Dummy links 0
;22 Dummy links Dummy links 0
;50 Transit-Only Links Transit-Only Links 0
;99 Centroid connectors Centroid connector 0
Page 2 (VOYAGER NETWORK)
MidRegion Council of Governments
--------------------------------
; collapse FTs into category-groups
GROUP = CAT_GROUP(1,FT)

;------------------
; prepare table to compute to flag oneway/twoway links in the network
;flip A and B nodes
BALINK = B*100000+A


NETWORK Stack Size = 696 bytes.
................................................................................


D:\ABQ_Model_72_pro_updated\02_APPLICATIONS\03_INITIALIZE_LOOP\00_ZERO_VOLUME_DELAY\ZVNET00A.NET (VOYAGER):

NET PGM=NETWORK (v.06/18/2018 [6.4.4 x64]) DATE=Wed Oct 28 10:05:39 2020
ID=
PAR Zones=1000 Nodes=10021 Links=13609 NodeRecs=5744
NVR 16 N X Y NORIG UI1 UI2 UI3 RAILRUNNER=017
RAILRUNNERFZ PARKSPACES CENTROID CONTROL=007 CONTROLID N_OLD
NORIG_OLD DASZ
LVR 27 A B AWDT12 VPP12_AM VPP12_PM CAPMOD
RIVERCROSSING DIRCOGID7 OBJECTID SCREENLINE DISTANCE
CATEGORY NLANES FUNCTION ISAUTO ISWALK ISPNRTOBUS AMPA
COGID SUBAREA PTSPD FT LEGACYCAT SRC=009 PDN FHWA_FUNC
ID=011


F(143): LOOKUP #1: LOOKUP=FT or RESULT=GROUP, variables not valid.

NETWORK ReturnCode = 2 Elapsed Time = 00:00:00
################################################################################

  • In the FILEI statement where you are reading in the CSV file, you have to define the field names. In a DBF file, the field headers and field types (number, text) are included in the file, so you do not have to define this. But when reading in a CSV file, you have to tell the voyager program what the field names and types are. Below is an example:

    FILEI LOOKUPI[1]="FT_CAT_Groups.csv" FT=#1, GROUP=#3, FTNAME(C)=#4

    In the above example, 'FT' is the name I am assigning to the first field, 'GROUP' is the name I am assigning to the third field and 'FTNAME' is the name for the fourth fields. In this case, I am ignoring the second field since I do not need it in this script. So you can choose which fields you want to read in. Also the notation FTNAME(C) indicates that this is a character field (enclosed in double quotes). By default all fields are assumed to be numbers.

  • Follwing your advise, seems that cube is still not able to read the file

  • There are two different issues here. One is using CSV files with Voyager for which I provided you the updated script earlier.

    For opening CSV files from Application Manager, you can do it two ways:

    1. Right click on the file box and select 'Text Edit' which should open the CSV file in the text editor.
    2. Go to Application Tools and then make sure that the CSV Editor option is set to Excel (default) to open CSV files in MS Excel, or to C:\Program Files (X86)\Citilabs\Cube\Cube.exe to open the file in the CUBE text editor. This option determines what happens when you double click on a file box in application manager which has a .csv file.Screenshot below.

  • Reading the file as you are telling me is producing an error. 

    FILEI LOOKUPI[1] = "D:\ABQ_Model_72_pro_updated\01_Inputs\common\FT_CAT_Groups.csv" ft=#1,group=#2
    ** *****
    F(018): ft is invalid key
    F(018): group is invalid key

    FT_CAT_Groups.csv

    ft group
    0 0
    1 0
    3 4
    4 4
    5 3
    6 3
    7 2
    8 6
    9 0
    10 0
    11 1
    20 0
    50 0
    2 5
    13 4
  • I took a another look at the script and just realized that this is a FILEI LOOKUPI statement where you are trying to read in a CSV file. In this case, we cannot define field names. But you can directly use the field index in the LOOKUP statement. I updated your FILEI LOOKUPI and the LOOKUP statements as below. You should also make sure that the CSV file does not have any field headers. Otherwise, Voyager will throw an error.

    FILEI LOOKUPI[1] = "D:\ABQ_Model_72_pro_updated\01_Inputs\common\FT_CAT_Groups.csv"
    ;-------------------------
    LOOKUP LOOKUPI=1, NAME=CAT_GROUP,
    LOOKUP[1]=1, RESULT=2, ; CATEGORY GROUP ID
    INTERPOLATE=F, FAIL[1]=0, FAIL[2]=0, FAIL[3]=0

    Answer Verified By: daniel jimenez