Bentley Communities
Bentley Communities
  • Site
  • User
  • Site
  • Search
  • User
ProjectWise
  • Product Communities
ProjectWise
ProjectWise Design Integration Forum Full text Indexing of more than 1 Million Files in same datasource failed PJW SS4 590
    • Sign In

    • State Not Answered
    • Replies 2 replies
    • Subscribers 61 subscribers
    • Views 1117 views
    • Users 0 members are here
    • ProjectWise
    • SS4
    • indexation

    Full text Indexing of more than 1 Million Files in same datasource failed PJW SS4 590

    Ludovick
    Offline Ludovick over 6 years ago

    Hi,

    On one of my biggest datasource we have more than 1 Million Files,

    With full text indexation ,  Pjw create text files with guid of the document on folders (one by datasource) which then are process by microsoft search server

    FTR extraction seems to be done in two pass : one (about 2Kb result) for only some metadata as the filename, etc ... and on the second pass the result file grows up with the extraction of the text of the document.

    On this big datasource the first pass semms to be ok, i can find a 2 kb file for each of my word documents for example.

    But the second pass seems to never hapend so i haven't the full text indexation of my files.

    It's not about the files thereselves, full text Indexation on some of the same office / cad files on a smaller datasource works fine.

    So, is there a limitation with the number of files which can be processed ?

    FYI this datasource has been upgraded from a projectwise SS3 Datasource

    Using  :  

    - PJW Explorer SS4 590 ou connect

    - Integration Server PJW SS4 590 (cluster of two) on Windows 2012 Server

    - Full text Indexation done by a third integration /indexing server PJW SS4 590 using MS Search Server on windows 2012 Server

    • Sign in to reply
    • Cancel
    • Kevin van Haaren
      0 Offline Kevin van Haaren Wed, Sep 13 2017 2:38 PM

      Highly, highly recommend you get in contact with Bentley support. Indexing is one of the trickiest pieces of projectwise to get up and running correctly and keep it that way.

      • We have 1 datasource with 2.5 million documents in it.
      • The total document count for all our datasources is around 8.8 million documents
      • We have 2 index servers that handle all our datasources, one that handles thumbnails/file property extractions, the 2nd is full text extractions (these are not clustered, they run independently)
      • We're running ProjectWise 10.00.01.67 on Windows 2012 servers

       

      • Cancel
      • Vote Up 0 Vote Down
      • Sign in to reply
      • Verify Answer
      • Cancel
    • Ludovick
      0 Offline Ludovick Fri, Sep 22 2017 9:36 AM in reply to Kevin van Haaren

      Thanks Kevin for your answer and advises.

      I think i finally found a solution , FTR indexation is running now on this datasource (+100.000 done at this time in few days)

      FTR seems to be stuck, then i I forced FTR reprocess only a small folder at the top of the hierarchy then all seems going on now ....

      -

      Ludovick LAGREVOL

      Projectwise Administrator

      • Cancel
      • Vote Up 0 Vote Down
      • Sign in to reply
      • Verify Answer
      • Cancel

    Communities
    • Home
    • Getting Started
    • Community Central
    • Products
    • Support
    • Secure File Upload
    • Feedback
    Support and Services
    • Home
    • Product Support
    • Downloads
    • Subscription Services Portal
    Training and Learning
    • Home
    • About Bentley Institute
    • My Learning History
    • Reference Books
    Social Media
    •    LinkedIn
    •    Facebook
    •    Twitter
    •    YouTube
    •    RSS Feed
    •    Email

    © 2023 Bentley Systems, Incorporated  |  Contact Us  |  Privacy |  Terms of Use  |  Cookies