Assurance Plans Guideline

All business continuity analysis should be risk based, and risk prioritised to deal with the important business risks first. This means that any risks to your business need to be identified, examined and dealt with.
There are 4 options for dealing with each risk:

1. Reduce the risk. Reducing the risk falls into 2 categories - reducing the likelihood of the problem occurring and reducing the impact of the problem if it does happen. A simple example is that by having a fire alarm you are reducing the likelihood of a fire spreading unseen and by installing a sprinkler system you are reducing the impact of fire.

Reducing the risk is often referred to as mitigation. For example, data backups are a form of mitigation. They reduce the impact if a problem occurs which affects the primary data source. Any mitigating actions require testing to provide assurance they work when required.

2. Transfer the risk. This is an interesting option which may be seen as a get-out, but which is a perfectly valid thing to do. By transferring a risk it becomes someone else’s problem and you therefore have the risk covered. We are not talking about blaming someone else, or even transferring the risk to someone else in the company.

For example, there could be a risk that office space will not be available in the case of a disaster in the main location. Therefore the risk can be transferred to a third party company which organises office space for disaster recovery and keeps offices available for companies who need such a recovery service.

3. Accept the risk. By accepting the risk of a potential problem you are at least aware of its existence and can plan for it happening. If it is a risk that would have no impact for an acceptable period of time it should still be noted but you may decide to take no action until it occurs.

Almost by definition, accepting a risk is also reducing the impact of the risk as you are aware of the potential problem and can write it into your business continuity plan.

4. Ignore the risk. This option should never be selected. There is never a reason for ignoring a risk once it has been identified. A risk can be accepted (acknowledged) but must never be ignored.

Once the actions for each risk have been identified, then anything put in place to help cope with a risk needs testing. However, many companies either test nothing at all or try testing every facet of a business continuity plan. Both methods are doomed to failure. The answer is to adopt a risk based testing approach from two perspectives: the business continuity plan is fit for purpose and it will work when invoked.

A health check (testing the plan is fit for purpose) needs to be performed by someone other than the authors of the business continuity plan. Ideally it’s performed by an independent third party that specialises in testing business continuity plans, but it could be a disinterested party from another part of the company. Independence is essential here for an objective assessment.

Testing the plan will work when invoked, must be viewed in a business context and the elements of the plan prioritised so that the risks with the most business impact and likelihood are tested first. This approach and the techniques to perform business continuity testing in a cost effective manner are the subject of other articles.

Copyright Acutest UK 2005

A Streeb is an experienced practitioner of business continuity testing at Acutest, an independent consultancy specialising in business continuity assurance and software testing services. For more information on this topic visit http://www.acutest.co.uk or send an email to enquires@acutest.co.uk

Share This

Better to fail for real than fail to really fail. Huh?
We know you’ve experienced this. Let’s say you just added some new functionality into your software, and you run a new build. And let’s say that 50% of your test cases fail. What is the first thing you assume?

We’ve asked this same question as our “teaser pitch” last winter to 100 developers and QA professionals who walked up to our booth at a recent conference, and 95 of them had the same answer! The tests must be broken!

This creates a cascading set of bad assumptions that will make your manager recite the adage about “ASS out of U and ME” on the whiteboard at the next project meeting. Here’s why.

• You assume that the problem is not with your application, it is with the test cases themselves being broken or no longer valid.
• So you spend time comparing the test cases with whatever changed in your new build.
• Then you dig into the test scripts to try to figure out why the test case is no longer passing, and rework them until they pass.
• Or you just give up and try validating by clicking through your old Word document test cases. Fun busy work.

How can you possibly call this testing? Rather than using the test to validate the application, you are using the application to test the test case - which is a program you coded!

Yes, unit tests are important for finding structural bugs in your code. But once a unit test tries to get beyond that granular level of testing, it becomes another fragile program in your development environment.

It is outrageous to assume that relying on coded unit test cases alone offers you any value in functional testing. In fact, the whole process is so manual and highly inefficient, that you wonder if you are doing anything more than making busy work for your own team.

Unit testing has its limits. There are methods people have tried to get beyond these limits, but it is like challenging the theory of gravity.

• Attempting to code for reuse - may seem possible but can only get you to the edge of Unit testing’s limits.
• Attempting to test the UI with your QA group, doesn’t really work if you can’t see those middle and back-end layers.

What makes false failures so dangerous? Besides the fact that they are a morale vampire that will make the team give up on testing, false failures impact the overall effectiveness of testing. If you don’t know if a failing test case is even valid, what do you really learn from testing? It is like a detective that never gathers evidence.

Time to declare war on false failures.

Jason has more than 13 years of experience in executing marketing plans, re-engineering business processes and meeting customer requirements for technology and consumer companies such as HP, IBM, EDS, Delphi, TaylorMade, Sun, Realm, Adaptec, Motorola, Sprint and currently with iTKO. Jason writes articles on a variety of subjects including software testing for http://www.itko.com.

Share This

It won’t matter how effective your WinRunner Team is if no-one outside your immediate organization knows about your accomplishments. For this reason, marketing your WinRunner Team is vital to your success. When times get tough, executives look for cost-cutting measures. The QA group is often the first on the chopping block. If these high level executives don’t fully understand and appreciate the value of your service, they will see the cost of WinRunner licenses and maintenance as well as the highly skilled, but also more expensive WinRunner engineers as a nice place to start trimming the budget. They will not have the time or luxury to launch an investigation to see if these services are really necessary.

The next thing you know, you have a nice library of WinRunner scripts, but no tools to execute them and no one with the skills necessary to modify the scripts as applications are updated. However, if these high level executives have personal knowledge about the benefits of software WinRunner in terms they understand, which are time and cost savings to the business they support, they will be less likely to put it on the chopping block.

High level executives don’t have the time or energy to seek you out and find out what benefits the automated testing has to offer to the organization. Instead, you must make the effort to seek them out. Marketing your WinRunner group is the responsibility of the entire team, but the heaviest burden lies upon upper-management and Vice Presidents because they have daily contact with peers at their level and higher.

Demonstrations for High Level Executives

One of the best ways to market your WinRunner team is to demonstrate to Executives, what you have done with WinRunner tool and how it helps their business. When you know an executive will be in town for a day, arrange for a thirty minute meeting. Executives are busy and everyone wants a piece of their time, so limit the time to thirty minutes. It will be sufficient to demonstrate what you have done with the product as well as how it benefits their business.

When you have confirmed that the executive will be able to attend a demonstration, it’s time to find a suitable conference room. Choose one that has live network connections and a screen to display the laptop image. Schedule the conference room for thirty to sixty minutes before the executive arrives in order to prepare. Use this time to set up the laptop, projector, and perform a dry-run of the script. Verify with the development engineering groups that no loads will be affecting the application you plan to demonstrate. Explain how important it is that nothing impact the environment, which will cause the application to go down during the presentation.

Invite as many members of the WinRunner team as possible so that the executive has the opportunity to meet everyone. However, if this is not possible due to the current work-load, invite only key individuals, preferably the ones who created or currently maintain the script that will be part of the presentation. They know most about it and will be able to troubleshoot any problems that arise more efficiently than someone who is not as familiar with the application or script.

Begin the meeting by making introductions and pass out an agenda so that everyone knows where the meeting is going and what will be covered. Give a brief overview of the application that will be demonstrated. The application should be one that the executive is familiar with and the script should run the length of the meeting (or longer). Ideally the application will have a lot of fields, making hand-typed data entry tedious. WinRunnerl will whiz through the application at an impressive speed.

While the script is running, explain how long it takes to manually run one test case verses how long it take WinRunner to execute one test case. Translate this into one test iteration so everyone can see how much time WinRunner saves on a weekly or monthly basis. Mention that the manual testers, who used to perform this testing, are now free to work on other projects, while this one is testing it’s self. At the end of the meeting, bring up the report to show how easy it is to identify which test cases passed and which, failed.

Executives are usually in back-to-back meetings, some of which run over their time limits. Less-important meetings, such as your presentation, may get rescheduled at the last minute. Don’t be discouraged. Simply reschedule the meeting for a later date. These presentations are not a waste of time. Executives who see the benefits of WinRunner and the cost savings will not only hesitate to cut your program out of the budget, but they will also inform their peers, who are struggling with long testing cycles, of your success. Ultimately, your success is their success.

Take Advantage of Status Reports

Status reports are one of the best ways to demonstrate to the business, on a weekly basis, how much time and money they are currently saving by automating the software testing. Status reports should contain the following figures:

Weekly hours saved per application
Year to date hours saved
Number of application automated
Number of scripts
Cumulative hours saved this week for all applications
Cumulative hours saved to date for all applications
Database or Spreadsheet of Project Statistics

Once the business and upper management gets wind of your WinRunner team’s abilities, be prepared for a windfall of questions. You will be asked over and over about the number of applications that have been scripted, time saved through automation, and a host of related questions. The best way to be prepared is to have a database or spreadsheet with your current project statistics on hand. Not only will you appear organized and efficient, but you will not have to drop your current activities to scramble for statistics. Your project database or spreadsheet should show general and application specific statistics.

Let Others Toot Your Horn

Executives who have had positive experiences with you in the past will spread the word when their peers complain about manual testing or show an interest in automating their software testing.

Beef Up Your QA Website

Most organizations have an internal website with sections dedicated to each group within the organization. If your QA organization doesn’t already have a website, it’s time to create one. A QA website can help you streamline activities such as a project queue that prioritized new projects, and conduct customer satisfaction survey’s, and announce your successes to the rest of the company.

Your QA website will do nothing for your PR unless other groups and organizations have to access it in order to interact with your team. You can begin forcing other groups to access your website by creating a project work queue, where they must complete a form in order to have their project entered into the QA work queue. This is your opportunity to lay down the rules rather than be forced to abide by their rules. There are specific facts that need to be clear before QA can prioritize and -assign resources to a new project.

Departmental home pages generally consist of the group mission statement and basic information. Once people have seen it a few times, they will skim right over it and with out a second thought. What a waste of space! Home Page real estate is the most valuable area of your website because it’s the one page that everyone sees. You best real estate should be reserved for facts that demonstrate your team’s success. It’s not that the team mission statement isn’t important, but rather that the mission statement belongs on another page or at the bottom of your home page, after the statistics. These statistics can be arranged in such a way that they display a running total of the cumulative time saved to date for each application and as a whole.

Town Hall Meetings

Take advantage of Executive Town Hall meetings, which are often used to update employees on the success of the business and visions Executives have for the future. They usually include many top level executives, who have connections in other business units who may be in need of automated testing. It’s not unusual for each Vice President to be asked to stand up and say a few words about their team’s current activities. This is a good opportunity to repeat some of your automated testing statistics or, in some cases, a quick presentation.

Don’t be afraid to contact executives, explaining your success with WinRunner and that you would like to share this with the organization during the Town Hall meeting. Executives rarely have the opportunity to see what is really happening in the “trenches” and will be pleased to hear about your success, especially when it is clearly a cost-cutting measure.

About the Author
Danna Henderson has helped many oranizations automate their software testing with complex, data driven scripts. For more information about successful automated testing with WinRunner, contact WinRunner and Software Test Automation Tools.

Share This