Hello,
We are thinking of using ProjectWise Connect Edition version 10.00.03.49 to store approx. 10 million files within a single datasource. Is this feasible or does ProjectWise have some limitations on how many files or how big of a datasource can be in size?
ProjectWise is installed on Windows 2016 Server and uses Microsoft SQL Server 2016.
Br,
Henri
Hi there, how about having approx 6,000 files with say 10 versions of each file in one folder?
when you have 60,000 files in one ProjectWise folder and you use ProjectWise Explorer, opening the folder in explorer just to view the files will take FOREVER. It has to return all that data for every single file. If you must have all 60,000 in one folder create a custom view with absolutely the minimum number of columns on it. Just Name would be best if you can.
Turning off "View All Versions" would be a big help here as it would reduce the number of visible files, and therefore the amount of data returned, by 10.
60k rows in a database with the same parent ID should not even make a database blip unless indexes are set up incorrectly. Granted, if you have a security nightmare and don't use best practices, that could make it horrible. So, assuming that's not an issue, what is PWExplorer doing to make it so slow? If you have slow servers or slow client machines, your mileage will be less.
Bill
.... a factor of 10
Generally we see issues on the network transfer of the data, and the time for the client to display it, not the database query. Depending on the number of columns you have visible it can add up to a sizable chunk of data.
For example if you have name, filename, description, file size, file modified, file modified by, status columns and assume around 10 characters each for character string, 8 bytes each for the datetime & file size, 1 byte for the status, it's about 57 bytes/row* or 3.3 Megabytes every time you bring up the listing. Assuming network compression is on it should reduce that by about 50%.
Network transfer, plus compression/decompression, sorting (I'm assuming sorting is done on the client, not the database), etc.. and it adds up to a notable delay when you click on the folder. Enough to get user complaints. A lot of them. Even reducing the columns down to just name, modified date and status was slow.
Our most populated folder is right at 58,000 files. It's been a number of years since that particular project was completed and back then the network speeds were much slower but we really got lot of complaints. (excluding this one datasource our typical max number of files is much closer to around 10,000 and we encourage using sub-folders at about 5-8,000.)
(* I ignored unicode encoding of the names since 99% of our file names & usernames are straight ASCII. I assumed file sizes are sent as 64 bit unsigned integers as PW Explorer supports files over 4GB in size)