Insights & News

Electronic Discovery and Content Management Discussions and Issues

When Black Swans Return

I was brought into a client site to help develop with Disaster Recovery for a content management system. My clients offices where in New York City, in lower Manhattan. The 9/11/2001 attacks prevented communications from their data center and their corporate offices. After a few days the communication channels were open and business was allowed to be conducted. 
This was considered a Black Swan type of risk and highly unlikely to reoccur. A Black Swan event as described by Nassim Nicholas Taleb is based on three criteria
  1. The event is a surprise (to the observer).
  2. The event has a major effect.
  3. After the first recorded instance of the event, it is rationalized by hindsight, as if it could have been expected; that is, the relevant data were available but unaccounted for in risk mitigation programs. The same is true for the personal perception by individuals.
The Swan reared its ugly head again in August 14th 2003 when New York City experienced a blackout. Again, the data center was unreachable. 
I was involved with the design of their enterprise content management system’s architecture. The system needed to accommodate 2.7 million workflows a day with 16 million documents. Therefore building two systems, production and backup, to manage the traffic and the workflow was a very expensive endeavor. 
The DR site chosen was in Jersey City, New Jersey, across the river from my client’s offices. Taking advantage of the close proximity of the DR site and the need for a large and powerful system, I suggested that we use the DR site in the design to process the workflows. The data would reside in NYC with failover to the New Jersey site in an active-passive configuration. We did discuss active-active, yet the costs were too much to overcome.

The workflows were going to be hardest part to design. Only one thousand concurrent workflows could be processed per Application server. I created five application service processes per physical server and replicated the same configuration at the DR site. Therefore the system could accommodate six thousand concurrent workflows. As designed the system was designed for 3 million workflows over a twenty-four hour period
Therefore turning Black Swans white.
The Questionnaire – the electronic discovery ident...
Marshalling Your Team

Related Posts


No comments yet