Bentley Communities
Bentley Communities
  • Site
  • User
  • Site
  • Search
  • User
ProjectWise
  • Product Communities
ProjectWise
ProjectWise PowerShell Extensions Forum Issues with Import-PWDocumentsFromArchive
    • Sign In

    • State Not Answered
    • Replies 7 replies
    • Subscribers 66 subscribers
    • Views 319 views
    • Users 0 members are here

    Issues with Import-PWDocumentsFromArchive

    Kevin van Haaren
    Online Kevin van Haaren 1 month ago
    1. The -InputFile option requires an absolute path, doesn't check current directory or support relative paths
      1. So can't do Import-PWDocumentsFromArchive -InputFile '.\mydocs.sqlite'
    2. The application set in the source is not set in the destination

    For 2, I created a folder in one datasource with different applications set for a lot of the files. Most are .dgn files with an application that is not the default one for the .dgn extension. But I also created 2 files with bogus extensions that don't have a default but I assigned them to an application anyway.

    I then restored the sqlite to a different datasource (I also tested restoring to a different folder in the same datasource with same result). The result is the applications are just assigned to whatever the default is for the extension or nothing if there is no default. I verified the destination datasource has the applications named exactly the same as the source.

    I verified the exported sqlite file does have the correct application names in it.

    Export command (forgot the -verbose though):

    Export-PWDocumentsToArchive -OutputFolder 'C:\code\_Tests\' -ProjectWiseFolder 'Application Test' -OutputFileName 'AppTestExport.sqlite3' -NoAuditTrail -ExcludeSecondaryAuditTrail -Overwrite
    Returned 0 matches
    WARNING: No rows in 'General_AuditTrail'. Not added.
    WARNING: No rows in 'References'. Not added.
    WARNING: No rows in 'FlatSets'. Not added.
    WARNING: No rows in 'FlatSetMembers'. Not added.
    WARNING: No rows in 'General_RichProjects'. Not added.
    WARNING: No rows in 'General_RichProjectProperties'. Not added.
    WARNING: No rows in 'General_ProjectTypes'. Not added.
    WARNING: No rows in 'General_ProjectProperties'. Not added.
    WARNING: No rows in 'General_Workflows'. Not added.
    WARNING: No rows in 'Environments'. Not added.
    WARNING: No rows in 'EnvironmentAttributes'. Not added.
    WARNING: No rows in 'Interfaces'. Not added.
    

    Import command (with -verbose):

    Import-PWDocumentsFromArchive -InputFile 'c:\code\_Tests\AppTestExport.sqlite' -DefaultStorage 'Hosted_Training' -Verbose
    VERBOSE: This is version 3 of the archive format.
    WARNING: Table Environments, EnvironmentAttributes or Interfaces is missing. Can't create missing environments if any.
    WARNING: Table General_Workflows is missing. Can't create missing workflows if any.
    WARNING: Tables General_ProjectTypes and General_ProjectProperties are missing. Can't create missing project types if
    any.
    VERBOSE: Folder table row count: 1.
    WARNING: General_AuditTrail missing, so no audit trail will be imported.
    VERBOSE: Document table row count: 9.
    VERBOSE: Total file parts in blob table: 1
    VERBOSE: Extracted ZIP
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\15c993bd-8f1e-476a-a979-6ce5c130557b.zip'.
    VERBOSE: Extracting
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\MicroStation_3D_Design_Test.dgn'.
    VERBOSE: Processing 1/9: 'Application Test\MicroStation_3D_Design_Test.dgn'.
    VERBOSE: Creating new document 'MicroStation_3D_Design_Test.dgn'
    VERBOSE: Total file parts in blob table: 1
    VERBOSE: Extracted ZIP
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\53340c23-8c74-4d90-a366-d9268b460067.zip'.
    VERBOSE: Extracting 'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\Test_File.fake'.
    VERBOSE: Processing 2/9: 'Application Test\Test_File.fake'.
    VERBOSE: Creating new document 'Test_File.fake'
    VERBOSE: Total file parts in blob table: 1
    VERBOSE: Extracted ZIP
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\60268e5f-f21a-4576-87ec-b7486c99d101.zip'.
    VERBOSE: Extracting
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\OpenBridge_3D_Design.dgn'.
    VERBOSE: Processing 3/9: 'Application Test\OpenBridge_3D_Design.dgn'.
    VERBOSE: Creating new document 'OpenBridge_3D_Design.dgn'
    VERBOSE: Total file parts in blob table: 1
    VERBOSE: Extracted ZIP
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\8eb1aeb9-0a76-418f-9960-9f91b1996bc0.zip'.
    VERBOSE: Extracting
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\OpenRoads_3D_Design_Test.dgn'.
    VERBOSE: Processing 4/9: 'Application Test\OpenRoads_3D_Design_Test.dgn'.
    VERBOSE: Creating new document 'OpenRoads_3D_Design_Test.dgn'
    VERBOSE: Total file parts in blob table: 1
    VERBOSE: Extracted ZIP
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\981a52a5-6c57-43e5-b75e-24115bd6384e.zip'.
    VERBOSE: Extracting 'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\Test_File_2.fake2'.
    VERBOSE: Processing 5/9: 'Application Test\Test_File_2.fake2'.
    VERBOSE: Creating new document 'Test_File_2.fake2'
    VERBOSE: Total file parts in blob table: 1
    VERBOSE: Extracted ZIP
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\a3822ee7-df38-49dc-b7ae-d0a7656e2a6c.zip'.
    VERBOSE: Extracting
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\OpenRail_3D_Design_Test.dgn'.
    VERBOSE: Processing 6/9: 'Application Test\OpenRail_3D_Design_Test.dgn'.
    VERBOSE: Creating new document 'OpenRail_3D_Design_Test.dgn'
    VERBOSE: Total file parts in blob table: 1
    VERBOSE: Extracted ZIP
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\c28c06cd-f862-4e5e-b892-b72aad31152c.zip'.
    VERBOSE: Extracting
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\OpenRoads_2D_Design_Test.dgn'.
    VERBOSE: Processing 7/9: 'Application Test\OpenRoads_2D_Design_Test.dgn'.
    VERBOSE: Creating new document 'OpenRoads_2D_Design_Test.dgn'
    VERBOSE: Total file parts in blob table: 1
    VERBOSE: Extracted ZIP
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\c981a0cb-e71d-45b7-82f3-d8e0cc664b96.zip'.
    VERBOSE: Extracting
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\MicroStation_2D_Design_Test.dgn'.
    VERBOSE: Processing 8/9: 'Application Test\MicroStation_2D_Design_Test.dgn'.
    VERBOSE: Creating new document 'MicroStation_2D_Design_Test.dgn'
    VERBOSE: Total file parts in blob table: 1
    VERBOSE: Extracted ZIP
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\ce96a25a-f4a7-4826-bb8f-fcaf2869f987.zip'.
    VERBOSE: Extracting
    'C:\Users\kvanhaaren\AppData\Local\Temp\ad2b6919-fac8-4939-8a77-66bb2d3d6d1d\OpenRail_2D_Design_Test.dgn'.
    VERBOSE: Processing 9/9: 'Application Test\OpenRail_2D_Design_Test.dgn'.
    VERBOSE: Creating new document 'OpenRail_2D_Design_Test.dgn'
    WARNING: No References table found.
    VERBOSE: Imported 9 documents from c:\code\_Tests\AppTestExport.sqlite.

    • Sign in to reply
    • Cancel

    Top Replies

    • Kevin van Haaren
      Online Kevin van Haaren Thu, Oct 26 2023 12:23 PM in reply to Robert McMillan +1
      When I use the commands in scripts i do similar (although Convert-Path will convert relative paths to full paths for you instead of manually doing it. You can cheat on the parameters, although I rarely…
    Parents
    • MWBSI
      0 MWBSI Thu, Oct 19 2023 7:17 PM

      Thanks for reporting the issue about the wrong applications being created.  We have confirmed the issue and it should be fixed next release.  As for not supporting relative paths we're not going to be doing anything about that any time soon, assuming it's really the case.  Please use Set-Location to confirm the current directory is really what you think it is.   Yes, the cmdlet checks whether the input SQLite file actually exists and this check seems to work in the vast majority of cases.  It issues a warning and quits when the input file does not exist. I'm not sure we'll handle the multiple extensions either Slight smile

      Mark Weisman | Bentley Systems 

      • Cancel
      • Vote Up 0 Vote Down
      • Sign in to reply
      • Verify Answer
      • Cancel
    Reply
    • MWBSI
      0 MWBSI Thu, Oct 19 2023 7:17 PM

      Thanks for reporting the issue about the wrong applications being created.  We have confirmed the issue and it should be fixed next release.  As for not supporting relative paths we're not going to be doing anything about that any time soon, assuming it's really the case.  Please use Set-Location to confirm the current directory is really what you think it is.   Yes, the cmdlet checks whether the input SQLite file actually exists and this check seems to work in the vast majority of cases.  It issues a warning and quits when the input file does not exist. I'm not sure we'll handle the multiple extensions either Slight smile

      Mark Weisman | Bentley Systems 

      • Cancel
      • Vote Up 0 Vote Down
      • Sign in to reply
      • Verify Answer
      • Cancel
    Children
    • Kevin van Haaren
      0 Online Kevin van Haaren Mon, Oct 23 2023 5:15 PM in reply to MWBSI

      Thanks on the applications, it'll be a big help.

      I suspect what's happening on the the path is that I use tab completion to fill in the name of sqlite file from the local directory. In powershell when you do this it makes the path to the file .\filename, adding the .\ makes it a relative path and that fails. I have not tried using a filename without the .\

      Out of curiosity, is there a code or security or just time reason for not supporting relative paths? I assumed (famous last words) the .net GetFullPath made it easy to convert a relative path to a full path and then check that.

       

      • Cancel
      • Vote Up 0 Vote Down
      • Sign in to reply
      • Verify Answer
      • Cancel
    • MWBSI
      0 MWBSI Mon, Oct 23 2023 9:15 PM in reply to Kevin van Haaren

      Ok, the fact that tab justification yields relative paths makes this worth investigating some time in the future.    It isn't obvious why the cmdlet code is not supporting relative paths.  It is not explicitly rejecting them.  As for support of relative paths being a security issue -- food for thought (and the prioritization of the issue).  Perhaps PowerShell should not have tab justification Slight smile

      Thanks,

      Mark Weisman | Bentley Systems

          

      • Cancel
      • Vote Up 0 Vote Down
      • Sign in to reply
      • Verify Answer
      • Cancel
    • Kevin van Haaren
      0 Online Kevin van Haaren Tue, Oct 24 2023 11:09 AM in reply to MWBSI

      I would die with out tab completion! Powershell doesn’t support the old windows behavior for executing things in the current directory automatically when no path is provided which probably fixes a bunch of exploit. Tab completion adding the .\ automatically is probably a partial fix as it shows you exactly which file will be used.

      Not a big deal pwps_dab doesn’t support it yet.

      thanks again

       

      • Cancel
      • Vote Up 0 Vote Down
      • Sign in to reply
      • Verify Answer
      • Cancel
    • Robert McMillan
      0 Offline Robert McMillan Tue, Oct 24 2023 6:33 PM in reply to Kevin van Haaren

      I've wrapped a number of the PWPS_DAB cmdlets with my own functions named something like Import-PWDocumentsFromArchive2 to add some pre/post process functionality and extend the original cmdlet. Might be an option to do that and add your own relative path tab completion handling in the interim although admitedly that particular cmdlet has a few more switches than normal to passthrough correctly.

      • Cancel
      • Vote Up 0 Vote Down
      • Sign in to reply
      • Verify Answer
      • Cancel
    • Robert McMillan
      0 Offline Robert McMillan Tue, Oct 24 2023 7:52 PM in reply to Robert McMillan

      Code is untested but this is the idea.

      function Import-PWDocumentsFromArchive2 {
          <#
          .SYNOPSIS
              Imports files, metadata and audit trail info from a SQLite archive container generated by Export-PWDocumentsToArchive.
      
          .DESCRIPTION
              Import folders, files, metadata, and audit trail from a SQLite database.
      
          .PARAMETER InputFile
              Path and SQLLite file to import from. Will not look for it in local folder by default.
      
          .PARAMETER TargetProjectWiseFolder
              Target parent folder. Builds relative folders beneath.
              
          .PARAMETER DefaultStorage
              Storage for new folders. If not included, the parent folder's storage area is used.
              
          .PARAMETER ReplaceIfExists
              If file exists in ProjectWise, replace it. Will version if not set.
              
          .PARAMETER LocalFolder
              Local folder for extracting files.
              
          .PARAMETER EnvironmentsOnly
              Just import environments.
              
          .PARAMETER SkipDocumentAttributeUpdate
              When included, document attributes will NOT be updated.
                      
          .PARAMETER SkipSecondaryAttributeUpdate
              When included, a secondary attempt to update individual document attributes will not be performed.
              
              This secondary attempt to update document attributes ONLY occurs when the initial attempt to update ALL document attributes fails.
      
          .PARAMETER RetainFileNames
              When included, if versions of being created, the filename of the source file will NOT overwrite the existing file name.
      
          .PARAMETER ExcludeEnvironments
              Don't import environments.
              
              If the environments are already in your datasource, use this switch.
      
          .PARAMETER UpdateExistingEnvironments
      
          .NOTES
              Name: Import-PWDocumentsFromArchive2.ps1
              DateCreated: 2023-10-25
      
          .EXAMPLE
              ----------  EXAMPLE 1  ----------
          
              Will import documents from an archive container.
              Import-PWDocumentsFromArchive -InputFile 'c:\export\export.sqlite' -TargetFolder "Imports\From Disk" -DefaultStorage Storage01 -ReplaceIfExists
          
              ----------  EXAMPLE 2  ----------
          
              Will import documents from an archive container. Uses a local folder and the document attributes will not be updated.
              Import-PWDocumentsFromArchive -InputFile 'c:\export\export.sqlite' -TargetFolder "Imports\From Disk" -LocalFolder 'd:\temp' -SkipDocumentAttributeUpdate -ReplaceIfExists
          #>
      
          [CmdletBinding()]
          Param (
              [Parameter (Mandatory, Position = 0)]
              [ValidateNotNullOrEmpty()]
              [String] $InputFile,
      
              [Parameter (Position = 1)]
              [ValidateNotNullOrEmpty()]
              [String] $TargetProjectWiseFolder,
      
              [Parameter (Position = 2)]
              [ValidateNotNullOrEmpty()]
              [String] $DefaultStorage,
      
              [Parameter (Position = 3)]
              [Switch] $ReplaceIfExists,
              
              [Parameter (Position = 4)]
              [ValidateNotNullOrEmpty()]
              [String] $LocalFolder,
      
              [Parameter (Position = 5)]
              [Switch] $EnvironmentsOnly,
              
              [Parameter ()]
              [Switch] $ExcludeEnvironments,
      
              [Parameter ()]
              [Switch] $RetainFileNames,
      
              [Parameter ()]
              [Switch] $SkipDocumentAttributeUpdate,
      
              [Parameter ()]
              [Switch] $SkipSecondaryAttributeUpdate,
      
              [Parameter ()]
              [Switch] $UpdateExistingEnvironments
          )
      
          Begin {
              Write-Verbose $MyInvocation.MyCommand.Name
      
              if (!(Get-PWCurrentDatasource -Verbose:$false)) {
                  Write-Error 'Not logged into a ProjectWise datasource.'
                  break
              }
          }
      
          Process {
              $Splat = @{
                  InputFile                   = if ($InputFile.StartsWith('.\')) { $InputFile.Replace('.\', "$((Get-Location).Path)\") } else { $InputFile }
                  #TargetProjectWiseFolder
                  #DefaultStorage
                  ReplaceIfExists              = $ReplaceIfExists
                  #LocalFolder
                  EnvironmentsOnly             = $EnvironmentsOnly
                  ExcludeEnvironments          = $ExcludeEnvironments
                  RetainFileNames              = $RetainFileNames
                  SkipDocumentAttributeUpdate  = $SkipDocumentAttributeUpdate
                  SkipSecondaryAttributeUpdate = $SkipSecondaryAttributeUpdate
                  UpdateExistingEnvironments   = $UpdateExistingEnvironments
              }
              if ($TargetProjectWiseFolder) { $Splat.Add('TargetProjectWiseFolder', $TargetProjectWiseFolder) }
              if ($DefaultStorage)          { $Splat.Add('DefaultStorage'         , $DefaultStorage) }
              if ($LocalFolder)             { $Splat.Add('LocalFolder'            , $LocalFolder) }
      
              return Import-PWDocumentsFromArchive @Splat
          }                      
      
          End {}
      }
      

      • Cancel
      • Vote Up 0 Vote Down
      • Sign in to reply
      • Verify Answer
      • Cancel

    Communities
    • Home
    • Getting Started
    • Community Central
    • Products
    • Support
    • Secure File Upload
    • Feedback
    Support and Services
    • Home
    • Product Support
    • Downloads
    • Subscription Services Portal
    Training and Learning
    • Home
    • About Bentley Institute
    • My Learning History
    • Reference Books
    Social Media
    •    LinkedIn
    •    Facebook
    •    Twitter
    •    YouTube
    •    RSS Feed
    •    Email

    © 2023 Bentley Systems, Incorporated  |  Contact Us  |  Privacy |  Terms of Use  |  Cookies