Beginner’s Guide to Recovery Exercises

Richard Long

What business continuity or disaster recovery exercises have your performed? Do you know the difference and have distinct goals for them? Do you even do exercises? We receive many questions on this topic, to the point where we thought it might be helpful to devote today’s post to a “Beginner’s Guide to Recovery Exercises.”

In this overview, we’re going to provide some introductory information on this essential topic. Specifically, we’re going to answer the following questions:

  • Why perform recovery exercises?
  • What are the benefits of performing recovery exercises?
  • What are the three main types of exercises?
  • How do you develop a testing program?
  • What are the two areas for which recovery exercises are performed?
  • Are exercises for business continuity and IT recovery always performed separately?
  • What exercises should your organization be performing?
  • Do companies graduate beyond the need to do tabletop exercises?
  • How frequently should the different types of exercise be performed?

Although we’ve titled this a “beginner’s” guide, we believe much of the information will also be helpful to those with more experience in conducting recovery exercises. We hope those folks will read on – we believe they will find the post informative. As we have said in previous posts, frequent reminders are important to ensure we stay on top of important aspects of BCM.

Now we’ll take the above questions one by one, starting from the top.

Why perform recovery exercises?

If you are not performing exercises to make sure you can recover your business in the event of a disruption or disaster, then you have no way of knowing that the recovery plans and strategies you have put in place will actually work if called upon. You only hope they will work. If this is sufficient for you and your organization, you have no need to perform recovery exercises. However, if you would rather base the future of your organization on something stronger than hope, you should be performing such exercises. The good news is, it’s never too late to begin.

Remember, hope is not a strategy.

What are the main benefits of performing recovery exercises?

There are several benefits of doing recovery exercises. They are:

  1. Validating your recovery strategy.
  2. Validating your recovery processes.
  3. Identifying gaps in both your recovery processes and strategy allowing you to correct them before a real event occurs.
  4. Training your staff so people know what to do. Actions performed during a recovery are different than day to day activities. We need to practice.

What are the three main types of exercises?

Recovery exercises can be divided into three main types. They are:

  1. Tabletop exercises. The starting point for any testing program. These are walkthroughs of the plans to validate procedures without performing any actions. Think of this as a read through to validate procedures or identify any major issues or gaps.
  2. Simulated recovery exercises. Recovering applications and processes without impacting production access or work. The vast majority of exercises performed fall into this category.
  3. Production recovery exercises. In this type of exercise, you perform the recovery – taking production down and then running it out of the recovered environment, just as you would in a real-life disaster. This is much more rare and should only be performed once your tabletop and simulated recovery exercises have demonstrated full capability with very few or only minor issues.

How do you develop a testing program?

Start small and ramp up. The tests you run are based on the maturity of your program. Each of the tests is a kind of training before you move on to the next level. At each level, the stakes are higher, and the activity more closely replicates the situation of an actual disaster. Each level provides feedback and an opportunity to improve your procedures.

Start with a limited scope. You could do individual tabletop exercises with each department first. Then bring in multiple departments where dependencies exist. Make sure your processes and strategy seem sound before you go on to a simulated recovery.

Even within the different types of exercises, you’ll need to progress over time. With simulated recovery, for example, you’ll want to start with a few applications or business units then ramp up as you become more proficient.

Very few organizations actually perform a production recovery exercise. It’s too risky and most organizations have not done the necessary planning and preparation, which comes through performing the other two exercises multiple times over the course of many years. Even then, be sure to consider the risk to impacting production if something goes wrong.

What are the two areas for which recovery exercises are performed?

The areas for which recovery exercises are performed are:

  1. Business Continuity. These are the procedures for recovering the parts of the business other than information technology—so, departments such as finance and human resources and the rest: everything not IT. These folks’ focus during these exercises will be drilling on what they would do during an event to keep things going. Will they leave the facility and take equipment with them? Work at home? Make phone calls to customers or other employees? There may be IT or technology components, but these are things like phones, workstations, printers, etc. – anything other than applications or processing that occurs in the data center.
  2. Disaster Recovery. This is the data center IT side. This is all about recovering the applications and technology that support the business.

Are exercises for business continuity and IT recovery always performed separately?

In the beginning, probably. Mature organizations will exercise both of these areas together at some point. You’ll declare a mock disaster and while the IT team performs the recovery of your apps and technology, the business team will be performing their functions. These integrated exercises take a lot more planning than just working on one side or the other. The teams involved will need to talk about dependencies and the scope of the exercise must be clearly defined.

It’s often the case that an organization’s testing is more mature in one area than another. Typically, the IT side (DR) is ahead of the business side in terms of preparedness, processes, and documentation. It might happen that in your joint exercise the DR team, for example, performs a simulated recovery while the BC side does only a tabletop exercise.

An integrated exercise is something you work your way toward over time. Once you’ve done tabletops and increased the scope to include multiple apps and environments (this could require five or more smaller exercises), then you could consider an integrated test where you bring DR and BC together and run a combined exercise that leverages both plans. For a company just starting an exercise program, such a project might be two to five years down the road.

What exercises should my organization be performing?

This mostly depends on where you are in your exercise program. You have to walk before you can run. If you’ve yet to do tabletops, that’s the place to start. When you have validated your strategies through tabletop exercises, you’ll be ready to move on to simulated recovery exercises and maybe to production recovery exercises. When you have substantial experience in testing both sides of your business (BC and DR), you will be ready to think about conducting an integrated exercise as described above.

Do companies graduate beyond the need to do tabletop exercises?

Nope. Even companies with mature exercise programs can reap dividends from performing tabletop exercises. They are an underused resource in our exercise methodology. They’re easy to schedule and perform, take very little time, and bring ongoing benefits. Even after we learn to run, we still find it advantageous to walk a good deal of the time. These exercises are a good way to keep people thinking about DR/BC as well as to verify changes to plans, strategies, and processes when significant changes occur in the IT or business functions.

How frequently should the different types of exercise be performed?

Tabletop exercises performed within a single department you can do as time and resources allow. If you were doing a separate exercise for each department, you might do them quarterly.

Most organizations will do one to two major exercises a year. As you increase the scope of your exercises, they become more difficult to coordinate and execute. Depending on your strategy, you might do a smaller scale exercise once a quarter with more major scoped exercises annually. Exercises demonstrating your overall recovery strategy for both BC and DR should be performed at least annually. Depending on your recovery strategies, you may be able to perform smaller exercises much more frequently.

We hope you found this “Beginner’s Guide to Recovery Exercises” to be helpful, whether you are new to BCM and exercises or if it provided some reminders for those of you who are more experienced. In future posts, we’ll leverage some of our webinar material to delve into some of the topics covered above in greater detail.

In the meantime, please feel free to email us with your questions.

Aligning your ITIL Process with your Disaster Recovery Planbusiness emergencies