ProjectWise Performance Report

Just trying to get an understanding of others' usage. How often are you scheduling this to run? 

My plans are to have this run on every office cache box, however, should this be once a day? a few times a day?  Any other thoughts or tips on use would be greatly appreciated. 

Thanks and well done on this. 

...not confirmed 100% as the cause yet, but I've had to disable scheduled tasks to run the scripts on the same day as I implemented, as it's suspected of dragging our overall performance down. ;-(

I think there are some good points to be made here, and I’m glad the topic has been raised – long post incoming.
Before we get into it, be aware I have some updates both to the script and Power BI report coming soon.

An example…

Imagine we are trying to monitor the performance of a train line.

To do this, someone is sent to a train station once daily to ride the train and record metrics regarding their experience – how long they were waiting at the station, passenger levels on the train, travel time to destination, time taken to alight, etc. This monitoring is unlikely to cause any issues for the regular commuters.

Now consider that instead of one person, we send 100 people, and instead of once a day, they are now sent every hour. How will this affect the train service?

You may have a modern station and a fast train with numerous carriages designed to handle this load, so there is little or no effect on regular commuters. Conversely, your station might be old and not equipped to deal with the traffic, and your train might be slow or have not enough carriages. Now we have started to frustrate our regular commuters.

What if we sent 1000 people each to six stations on the same train line, every 10 minutes? What if we did this on the weekend as opposed to peak hour? Would that information even be useful? What key component is the train line example missing?

ProjectWise is not a train line…

Bringing this back to ProjectWise and the tool at hand – let’s first describe what the script does.

1. Logs into the target ProjectWise datasource
2. Creates two test files of the specified size in the script user’s working directory
3. Checks the specified (or default) environment exists, creates the environment if it doesn’t
4. Creates a test folder in ProjectWise
5. Creates a destination subfolder within the test folder from the previous step
6. Updates test folder properties
7. Creates first document in the test folder
8. Searches for and returns first test document
9. Checks out first test document
10. Checks in first test document
11. Creates second test document in the test folder
12. Copies out the second test document
13. Clears the specified user’s working directory
14. Copies out second test document (2)
15. Copies out second test document (3)
16. Updates first test document attributes
17. Updates first test document properties
18. Copies first test document to destination subfolder
19. Removes first test document from destination subfolder
20. Removes the destination subfolder
21. Removes the first and second test documents
22. Removes the test folder
23. Logs out of ProjectWise

The script takes a ride on the ProjectWise train, acting just like a regular commuter, and records metrics along the way. Just like our train line, we need to be aware of what the underlying infrastructure can handle so we don’t upset the regular commuters.

Do you have enough bandwidth in your office to run the script once daily with a 10MB file? How about the bandwidth/throughput on your servers? Can your database handle the extra transactions? How about the disks on your storage area?

The script is just a regular commuter on the train, and a regular user in your ProjectWise system. If you were to login to ProjectWise Explorer as a single user and perform the same tasks listed above – albeit more slowly than the script can manage – would you degrade overall system performance? Whilst in the absence of more information I’d hesitate to say with certainty, it’s very likely there are serious problems if one extra user is going to tip over system performance.

Now if there were 250 scripted users connecting to the same datasource, using a 1GB file, connecting from 10 offices, every 10 minutes, that would likely be a different story. Which again begs the question, would the information gathered in that circumstance even be useful?

What we are still missing…

The most important thing we are missing in the above scenarios is a clear goal. What are the questions we want to answer, and what information do we need to inform those answers?

For example, are you wanting to:

• Diagnose an intermittent performance issue from a specific location?
• Map general performance baselines across your ProjectWise network?
• Chart long term trends?
• Assess the effect of caching?
• Stress test your system?
• Et cetera, et cetera, et cetera…

What data would you need to inform the answers to those questions?

This should have the biggest impact on how you decide to use the script.
Start small. Build up. Reach out if you have questions. A single extra user should not bring down your system.

Thanks for sticking with it if you're still reading. I hope the information can be of some help. Let me know if there are problems

I think there are some good points to be made here, and I’m glad the topic has been raised – long post incoming.
Before we get into it, be aware I have some updates both to the script and Power BI report coming soon.

An example…

Imagine we are trying to monitor the performance of a train line.

To do this, someone is sent to a train station once daily to ride the train and record metrics regarding their experience – how long they were waiting at the station, passenger levels on the train, travel time to destination, time taken to alight, etc. This monitoring is unlikely to cause any issues for the regular commuters.

Now consider that instead of one person, we send 100 people, and instead of once a day, they are now sent every hour. How will this affect the train service?

You may have a modern station and a fast train with numerous carriages designed to handle this load, so there is little or no effect on regular commuters. Conversely, your station might be old and not equipped to deal with the traffic, and your train might be slow or have not enough carriages. Now we have started to frustrate our regular commuters.

What if we sent 1000 people each to six stations on the same train line, every 10 minutes? What if we did this on the weekend as opposed to peak hour? Would that information even be useful? What key component is the train line example missing?

ProjectWise is not a train line…

Bringing this back to ProjectWise and the tool at hand – let’s first describe what the script does.

1. Logs into the target ProjectWise datasource
2. Creates two test files of the specified size in the script user’s working directory
3. Checks the specified (or default) environment exists, creates the environment if it doesn’t
4. Creates a test folder in ProjectWise
5. Creates a destination subfolder within the test folder from the previous step
6. Updates test folder properties
7. Creates first document in the test folder
8. Searches for and returns first test document
9. Checks out first test document
10. Checks in first test document
11. Creates second test document in the test folder
12. Copies out the second test document
13. Clears the specified user’s working directory
14. Copies out second test document (2)
15. Copies out second test document (3)
16. Updates first test document attributes
17. Updates first test document properties
18. Copies first test document to destination subfolder
19. Removes first test document from destination subfolder
20. Removes the destination subfolder
21. Removes the first and second test documents
22. Removes the test folder
23. Logs out of ProjectWise

The script takes a ride on the ProjectWise train, acting just like a regular commuter, and records metrics along the way. Just like our train line, we need to be aware of what the underlying infrastructure can handle so we don’t upset the regular commuters.

Do you have enough bandwidth in your office to run the script once daily with a 10MB file? How about the bandwidth/throughput on your servers? Can your database handle the extra transactions? How about the disks on your storage area?

The script is just a regular commuter on the train, and a regular user in your ProjectWise system. If you were to login to ProjectWise Explorer as a single user and perform the same tasks listed above – albeit more slowly than the script can manage – would you degrade overall system performance? Whilst in the absence of more information I’d hesitate to say with certainty, it’s very likely there are serious problems if one extra user is going to tip over system performance.

Now if there were 250 scripted users connecting to the same datasource, using a 1GB file, connecting from 10 offices, every 10 minutes, that would likely be a different story. Which again begs the question, would the information gathered in that circumstance even be useful?

What we are still missing…

The most important thing we are missing in the above scenarios is a clear goal. What are the questions we want to answer, and what information do we need to inform those answers?

For example, are you wanting to:

• Diagnose an intermittent performance issue from a specific location?
• Map general performance baselines across your ProjectWise network?
• Chart long term trends?
• Assess the effect of caching?
• Stress test your system?
• Et cetera, et cetera, et cetera…

What data would you need to inform the answers to those questions?

This should have the biggest impact on how you decide to use the script.
Start small. Build up. Reach out if you have questions. A single extra user should not bring down your system.

Thanks for sticking with it if you're still reading. I hope the information can be of some help. Let me know if there are problems

Cheers Zach, great explanation 

My pleasure. Feel free to email or call if you'd like to discuss implementation or interpretation!

Excellent analogy and explanation.