Disaster Recovery and ProjectWise

When talking about disaster recovery and ProjectWise luckily there are only a couple of priority items that need to be taken into consideration, these are the storage area(s) and database.  The reason these are the most important pieces to a ProjectWise system in relation to disaster recovery is because with these two pieces a ProjectWise system can essentially be rebuilt anywhere in the world, in fact this is what support uses to replicate a user environment.  If I were to rank which piece out of the two were more important I would definitely pick the database, reason being the database has internal counters and metadata that if lost could cause havoc.  Remember a copy of the file can be retrieved from a user's working directory or even a cache server if it has been updated since the storage area's last backup so this makes the storage area a little less important then the database.  The other pieces like the dmskrnl.cfgs and indexing catalog can always be rebuilt with no data loss, but if a storage area and/or the database are lost a user will most definitely experience data loss.  When it comes to backing up a storage area and database these should ideally be done in sync or very close to in sync since these two pieces have a direct correlation with each other.  With the storage area and database backed up successfully an organization will be able to bring their system back up and online relatively quickly as long as there is a ProjectWise system on the same version installed somewhere in the world.  I always recommend users to back them up at a location off site and have a ProjectWise system also installed off site to shorten the downtime associated with a disaster. 

If a user is in a Managed Services hosted environment where we host the datasource this is a task we take on.  When in a hosted datasource the ProjectWise database is backed up once an hour and the storage area(s) are once every 24 hours, so essentially a user will only ever lose 24 hours' worth of work with the database only be an hour behind.  This backup is all done by the Managed Services team automatically and stored on our servers, so this is entirely handled by this team. 

One other option a user has is to use the iCS for PDF's distribution feature.  We have some users who are using the "mothership" concept and using the distribution feature to replicate their datasource to another system off site on a schedule, this schedule can be done every 30 min with only the deltas being transferred.  The nice thing about this is that if a disaster were to happen only a DNS change is needed to point the clients to the new server which means they are up and running again with only a 30 min or less gap in data.  The cost of servers however needs to be taken into consideration with this option. 

Now that we have established what needs to be backed up for disaster recovery, the frequency of the backups needs to be addressed next.  When determining the frequency of backups an organization should take into consideration what the financial risks and magnitude of harm are when your ProjectWise system is lost.  When determining the frequency of backups an organization should ask itself "How much data loss am I ok with?" reason being each organization is different.  We have some ProjectWise users with large implementations that could lose upwards of $300k an hour if their ProjectWise is down while we have smaller implementations that would lose that much in a month's time.  So what recovery time is best for your organization?  That is a question that needs to be addressed either with the help of Bentley or internal to your organization, but either way they need to be scheduled regularly.  

One of my first horror stories in support, many years ago, was a user called in because they could not check out any files from their system.  After some investigation we discovered that there was a home baked customization that went rogue on the integration server and zeroed out all of the files in their storage area.  After some initial panic I told the user they were fine as long as they had a backup, which they told me they had not backed up the storage area in a month's time because the backup software was broken.  So in reply to that I said well at least you have your database backed up so we can go retrieve the files from user desktops and any caching servers.  Well that is when the real panic set in because the database was not backed up in over a month's time as well.  Overall this company lost a month's worth of work which ended up costing them a couple of million dollars.  So please make sure you are doing your backups on a regular basis so you do not end up like the company who lost a couple of million dollars and a month's worth of work.

If you have any questions to the above on disaster recovery please contact us at Bentley and we will be glad to assist you in determining when and how to back up your data.

Anonymous