When Disaster Strikes: Preparing for the Worst

April showers bring May flowers. But what if it rains through May, and rains even harder in June? No one wants to believe that rain could lead to a flood, but when you live in the Wyoming Valley you pay attention. We have a history of flooding here, so Solid Cactus had a plan. Frankly, after Katrina, don’t we all need a plan if disaster strikes? This is what happened to us and how we reacted. Hopefully some of our experience will be of value for you.

Solid Cactus is based in Wilkes-Barre, PA in the middle of the Wyoming Valley. Through that valley runs the Susquehanna River. Residents of this old coal town remember the devastation inflicted upon them in June 1972 when Hurricane Agnes flooded the entire valley. I was still in diapers, but to this day stories are told of “The Agnes Flood” and 20 foot-high water marks on buildings are a reminder of that disaster. The flood expanded the banks of the river from its normal 200 yards across to eight miles wide. It took years for the area to finally recover from “The Agnes Flood” and nobody ever wanted to go through it again. But 34 years later – almost right to the date – our worst fears were unfolding before our eyes.

After the 5th straight day of heavy rains in late June the creeks filled up, the river began to rise and we started to worry. Most of the Wyoming Valley, including Wilkes-Barre, is protected by a levee system designed by the Army Corps of Engineers after the Agnes flood. However, they had never been tested above 35 feet.

Wednesday morning, we huddled in our office reviewing our disaster plan and getting ready to implement it. Eyes were glued to the TV and ears to the radio. Forecasters predicted a crest 3 feet higher than the levees. The river was rising at an astonishing foot an hour Wednesday morning, and by noon, a mandatory evacuation of the Wyoming Valley was issued. It was time to activate our plan.

First, we insisted that those who live in evacuated areas leave. Second, we sent an email out to our entire customer base letting them know what was happening and providing them with a method to contact us while our offices were closed. Then the trucks arrived. Solid Cactus, a company which must have constant interaction with our clients, was about to go dark.

Our sister company, Neeps Inc. is located about a mile away on higher ground out of the flood plain. A fleet of box trucks arrived to pick up computers, monitors, important customer records and anything that we would need to conduct business in the event our offices were inaccessible for a period of time. Even Sydney the rabbit was moved upstairs. Up at the Neeps Distribution Center part 2 of our plan was about to unfold.

We dismantled our server room taking our Exchange server and any other mission critical equipment. A team of Cisco certified engineers was standing by at Neeps waiting to reconnect our communications network. A few months earlier, we invested in a Cisco voice over IP telephone system, and it paid off! In 3 hours our entire network and phone system was dismantled, reassembled, and fully functional at our backup location.

Across the street from Neeps is our company training center. Desks were assembled and a high-powered wireless link was on standby ready to provide phone and internet service between the DC and the training center. We planned for the backup office because we knew the evacuation would keep us out of the city for days. Our main concern was maintaining contact with our clients and because of the VOIP system we did it with only 3 hours of downtime.

The river slowed its rise and the levees held, but officials kept the evacuation order in effect. As part of our disaster plan, programmers and designers worked from home on computers they took from the office. Account managers and consultants worked from home maintaining contact with clients via cell phone while the executive team and department managers were at our temporary location. Another email went out to our clients advising them of the situation and providing contact information.

The best news came Friday morning when the river finally crested and everyone breathed a very loud sigh of relief. In a reverse of Wednesday, the fleet of Neeps vehicles arrived at Solid Cactus precisely at noon. Teams unloaded computers and equipment and staged them outside while others put the office back into shape. Once everything was unloaded and put into place, we moved the phone system and network equipment from Neeps back to Solid Cactus. Once again, downtime was less than 3 hours (Thank you Cisco!). The rest of Friday was spent re-grouping and making contact with our clients. By 5pm it was business as usual.

The events we went through were trying. The entire Solid Cactus team pulled together; followed the disaster plan and executed it flawlessly. The spirit of teamwork and the dedication everyone had towards Solid Cactus was evident. And our clients! So many of you sent email offering support for employees effected by flooding, temporary office space, wishes, prayers and words of encouragement. It was amazing. Thank you all!

While your office may not be near a river or in an area subject to natural disasters, you should still have a disaster plan in place. What if a fire were to break out? What if a water pipe burst and flooded your office? Imagine every thing that could go wrong. Solid Cactus planned ahead with a temporary location, network portability, transportation and communication. Take a few minutes to think of what would happen if a disaster were to strike your company, would you be ready? Plan now.

Don’t wait until your own “river,” whatever form it takes, begins to rise around you.