We are thinking of using ProjectWise Connect Edition version 10.00.03.49 to store approx. 10 million files within a single datasource. Is this feasible or does ProjectWise have some limitations on how many files or how big of a datasource can be in size?
ProjectWise is installed on Windows 2016 Server and uses Microsoft SQL Server 2016.
60k rows in a database with the same parent ID should not even make a database blip unless indexes are set up incorrectly. Granted, if you have a security nightmare and don't use best practices, that could make it horrible. So, assuming that's not an issue, what is PWExplorer doing to make it so slow? If you have slow servers or slow client machines, your mileage will be less.
.... a factor of 10
Generally we see issues on the network transfer of the data, and the time for the client to display it, not the database query. Depending on the number of columns you have visible it can add up to a sizable chunk of data.
For example if you have name, filename, description, file size, file modified, file modified by, status columns and assume around 10 characters each for character string, 8 bytes each for the datetime & file size, 1 byte for the status, it's about 57 bytes/row* or 3.3 Megabytes every time you bring up the listing. Assuming network compression is on it should reduce that by about 50%.
Network transfer, plus compression/decompression, sorting (I'm assuming sorting is done on the client, not the database), etc.. and it adds up to a notable delay when you click on the folder. Enough to get user complaints. A lot of them. Even reducing the columns down to just name, modified date and status was slow.
Our most populated folder is right at 58,000 files. It's been a number of years since that particular project was completed and back then the network speeds were much slower but we really got lot of complaints. (excluding this one datasource our typical max number of files is much closer to around 10,000 and we encourage using sub-folders at about 5-8,000.)
(* I ignored unicode encoding of the names since 99% of our file names & usernames are straight ASCII. I assumed file sizes are sent as 64 bit unsigned integers as PW Explorer supports files over 4GB in size)
Thank you all for your help in supplying me with a better understanding. It's very much appreciated. The following is another explanation provided:
"If you have too much in a folder, it will take longer to load. If you think about a folder like a saved search, if you have more stuff in the search, it will take longer to return the results."