Issues with Import-PWDocuments

I'm running into issues with Import-PWDocuments. We've been using it for some time now to mirror a server location into ProjectWise to provide select external access to a large set of information. The dataset is roughly 200K files, 50K folders, and the sync usually takes 30-40 minutes each night to verify and sync the delta. The problem we're running into is that the sync thinks it needs to sync the same ~700 files (plus whatever legitimate file additions / modifications there may have been).

I have checked the length of the overall paths (<261), the length of the filenames (<128), and the length of each of the folders (<64). I thought maybe it was an issue with a special character, or some weird file modification date/time discrepancy, but didn't see anything that stood out. I can provide the logs from my last month of sync operations, but would rather not include them in the post.

Thanks for any assistance you are able to provide.

Parents Reply
  • ,

    I happened to be speaking with a few minutes ago we discussed your issue and he noticed that your datasource is hosted by MaS and he suggested using the -ServerIsUtc switch as that was added specifically for MaS hosted systems a while back.

    Note that the actual switch name is slightly different than what is in the change log.

    You might want to try that switch to see if it fixes the issues that you are seeing.

    Dan Williams
    Solution Consultant
    Bentley Systems, Incorporated
    Portland, OR, USA (Pacific Time UTC-08:00)

Children
  • ,

    Here are the switches that we are using (in addition to -InputFolder, -ProjectWiseFolder, -DefaultStorage, and -DefaultEnvironment): 

    -ServerIsUtc -MakeProjectWiseMatchDisk -Overwrite -ExcludeSourceDirectoryFromTargetPath -Exclusions "thumbs.db", "*.lnk", "*.tmp" -Verbose

  • ,

    I have provided Dan with... well probably an overwhelming amount of information privately. Here's a genericized version of the guts of the script for the public thread though:

        $LocationOneFolders = Get-ChildItem -LiteralPath "D:\Location One\Folder One\Location One Match Folder\"
    
        ForEach ($LocationOneFolder in $LocationOneFolders) {
    
            If ($LocationOneFolder.Name.Substring(0,3) -match "^\d{3}$") {
                $countynum = [int]$LocationOneFolder.Name.Substring(0,3)
            }
            Else {
                $countynum = 0
            }
    
            If (($countynum -ge 1) -and ($countynum -le 105) ) {
                
                Import-PWDocuments -InputFolder $LocationOneFolder.FullName -ProjectWiseFolder "Master Sync Location\Location One Match Folder\$($LocationOneFolder.Name)" -ServerIsUtc -DefaultStorage "Sync" -DefaultEnvironment "ENV_Sync" -MakeProjectWiseMatchDisk -Overwrite -ExcludeSourceDirectoryFromTargetPath -Exclusions "thumbs.db", "*.lnk", "*.tmp" -Verbose
                
            }
        }
    
    
        
        $LocationTwoFolders = Get-ChildItem -LiteralPath "E:\Location Two\Location Two Match Folder\"
    
    
        ForEach ($LocationTwoFolder in $LocationTwoFolders) {
    
            If ($LocationTwoFolder.Name.Substring(0,7) -match "^(fy) \d{4}") {
    
    
                $SubFolders = Get-ChildItem -LiteralPath $LocationTwoFolder.FullName
    
                ForEach ($SubFolder in $SubFolders) {
    
                    If ($SubFolder.Name -match "(k|ka)( |-|)\d{4}( |-|)\d{2}" -or $SubFolder.Name -like "_Special Case") {
    
                        Import-PWDocuments -InputFolder $Subfolder.FullName -ProjectWiseFolder "Master Sync Location\Location Two Match Folder\$($LocationTwoFolder.Name)\$($SubFolder.Name)" -ServerIsUtc -DefaultStorage "Sync" -DefaultEnvironment "ENV_Sync" -MakeProjectWiseMatchDisk -Overwrite -ExcludeSourceDirectoryFromTargetPath -Exclusions 'thumbs.db','*.lnk','*.tmp' -Verbose
                        }
                    elseif ($SubFolder.Name -like "_Nested Project") {
                    
                        $NestedProjectFolders = Get-ChildItem -LiteralPath $SubFolder.FullName
    
                        ForEach ($NestedProjectFolder in $NestedProjectFolders) {
    
                            If ($NestedProjectFolder.Name -match "(k|ka)( |-|)\d{4}( |-|)\d{2}") {
    
                            Import-PWDocuments -InputFolder $NestedProjectFolder.FullName -ProjectWiseFolder "Master Sync Location\Location Two Match Folder\$($LocationTwoFolder.Name)\$($SubFolder.Name)\$($NestedProjectFolder.Name)" -ServerIsUtc -DefaultStorage "Sync" -DefaultEnvironment "ENV_Sync" -MakeProjectWiseMatchDisk -Overwrite -ExcludeSourceDirectoryFromTargetPath -Exclusions 'thumbs.db','*.lnk','*.tmp' -Verbose
                            }
    
                        }
    
                    }
    
                }
            }
    
        }

    Thanks Brian.