Technique with examples orthogonal array testing

Orthogonal array testing is combinatorial software testing which uses a systematic, statistical way of deriving test conditions and test cases. It is followed in places where there are a large number of combinations that should be tested and would be risky if we do not test them.

Technique:
The steps for using orthogonal array technique are:
STEP 1: Analyze the given test inputs or requirements and list out the variables that needs to be tested for interaction.
STEP 2: Determine the number of choices or values for each variable.
STEP 3: Locate an orthogonal array which has a column for each variable and values within the columns that correspond to the values for each variable.
STEP 4: Map the variables with their values on to the orthogonal array.
STEP 5: Each row in the table corresponds to a test condition or a unique test case.

Examples: Company A has designed a web page with three distinct sections (Top, middle, and bottom). These sections can be individually shown or hidden by the user. The browsers on which the webpage can be viewed with are internet explorer and firefox. The server used for launching the application can be IIS, Apache and weblogic. Create an orthogonal array to test the interactions of the different sections. Use orthogonal array to select the test conditions that would be required to test this scenario.

Solution:
STEP1: The variables that needs to be tested for interaction are sections, visibility, browsers and servers.
STEP 2: The number of choices or values for each variable is:

• Sections: Top, middle, bottom
• Visibility: shown, hidden
• Web browser: Internet explorer, firefox
• Servers: IIS, APache amd weblogic

STEP 3: The orthogonal array that can be used would be Lbase 9(3power 4)array.
STEP4 : Map the variables with their values on to the orthogonal array

State transition diagram in testing

State Transition Testing Technique:
Steps for using state transition testing are:
STEP 1: Understand the various states that the system , user, or object can be in, including the initial and final states. Examples of states can be 'User raising a purchase order' or 'leave request is accepted'.  State are represented as,

STEP 2: Identify transitions, events, conditions, and actions that can and can't apply in each state.

STEP 3: Use a graph or table to model the system. This graph or table also serves as an oracle to predict correct system behavior along with a requirement specification.

STEP 4: For each event and condition (i.e each transition) verify that the correct action and next state occurs.

STEP 5: Create test cases in such a way that all states are visited at least once, all events are triggered at least once and all paths are executed at least once. (i.e all transitions in the system are tested at least once).

Example for using state transition testing:

What is State Transition Testing

State Transition Testing:
State-Transition diagrams are an excellent tool to capture certain types of system requirements and to document internal system design. These diagrams document the events that come into and are processed by a  system as well as the system's responses. When a system must remember what happened before or when valid and invalid orders of operation exist, then state transition testing could be used. It is useful in situations when workflow modelling or dataflow modelling has been done, which mean that the system moves from one state to another.

State Transition diagrams uses states, events, actions and transitions of the system to be tested. Together with all these components, they represent how a system interacts with the outside world and process them.

Some of the terms use din state transition testing are:
State: A state is a condition in which a system is waiting for one or more events. States remember inputs the system has received in the past and define how the system should respond to subsequent events when they occur. These events may cause state transitions and or initiate actions. The state is generally represented by values of one or more variables within a system.

Transition: It represents a change from one state to another caused by an events.
Event: Something that causes the system to change state. Events may have parameters associated with them. Events can be independent or casually related.

Actions:  An action is an operation initiated because of state change. Often these actions causes something to be created that is outputs of the system.

Decision Table Testing Technique and Examples

Technique followed for decision table testing are follows:
STEP 1: Analyze the given test inputs or requirements an list out the various conditions in the decision table.
STEP 2: Calculate the number of possible combinations (Rules)
STEP 3: Fill columns of the decision table with all possible combinations (Rules)
STEP 4: Find out cases where the values assumed by a variable are immaterial for a given combination Fill the same by "don't care" symbol.
STEP 5: For each of the combination of values, find out the action or expected result.
STEP 6: Create at least one Test case for each rule. If the rules are binary, a single test for each combination is probably sufficient. Else if a condition is a range of values,, consider testing at both the low and high end of range.

Example problem for decision table testing:
Question: The following instructions were taken from a University Examination Application Form. Examine the and create a Decision Table to represent the University rules and thereby decide on the test cases to be designed.

1. A candidate appearing for the exam should have a minimum 80% attendance
2. Has attended 3 internal tests with an average of 10 or more marks or
3. Has attended 2 internal tests with an average of 15 or more marks and 
4. Has taken up the external examination and scored more than 35 marks.

If the above conditions are fulfilled, then the student can be considered as pass in that subject
Answer:
Based on the decision table technique, the decision table conditions and actions can be drafted as given below:
C1 = Condition 1 - Attendance greater than 80%
C2 = Condition 2 - Attended 3 internal tests with an average of 10 or more marks
C3 = Condition 3 - Attended 2 internal tests with an average of 15 or more marks.
C4 = Condition 4 - Cleared external exam and scored more than 35 marks
A1 = Action 1 - Pass
A2 = Action 2 - Fail
R1 = Rule 1- C1, C2, C4 are satisfied
R2  = Rule 2- C1, C3, C4 are satisfied

Note: list the remaining rule ie. R3 TO R6 accordingly
The total number of alternatives = 2 the power of number of conditions = 2 the power of four = 16

Decision Table General Format

Decision Tables are used to describe and analyze problems that contain procedural decision situations that are characterized by one or more conditions; the state of which determines the execution of a set of actions.

They record the complex business rules that a system must implement. This serves as a guide to create Test Conditions. The basic parts of a decision table are shown below:

The upper left portion of the format is called the stub quadrant, it contains statements of the conditions. Similarly, the lower left portion is called the action stub quadrant, which contains statement of the actions. The condition entry and action quadrants appear in the upper and lower right portions (conditions and entry) forms a decision rule.

The various input conditions are represented by the conditions 1 through m and the actions are represented by actions 1 through n. These actions should be taken depending o the various combinations of input conditions.

Each of the rules defines a unique combination of conditions that result in the execution (firing) of the actions associated with rule. All the possible combinations of conditions define a set of alternatives. For each alternative, a test action should be considered. The number of alternatives increases exponentially with the number of conditions, which may b expressed as 2 power of Number of Conditions. When the decision table becomes too complex, a hierarchy of new decision tables can be constructed.

Note:

• The actions do not depend on the order in which the conditions are evaluated, but only their values.
• All the values are assumed to be available simultaneously.
• Actions depend only on the specified conditions, not on any previous input conditions or system state.

Recommended:

Decision Table Testing Technique with examples

Boundary Value Analysis Example

Boundary Value Analysis:
Boundary value Analysis is the act of dividing the given input set by a relation into groups or classes which is treated the same by the module or which should produce the same result. Any data value within a class is equivalent in terms of testing, to any other value.

The steps for using boundary value testing are:

1. Identify the equivalence classes from the given requirements or scenario.
2. Identify the boundaries of each Equivalence Class.
3. Create a test case for each boundary value by choosing one point above the boundary, one point below the boundary and one point on the boundary.
4. From the results obtained for one member in an equivalence class extrapolate results for all values in that partition.

Examples:
Consider the same problem what we have seen in equivalence class partitioning.

Equivalence Class Testing Technique Examples

Equivalence class Testing:
Equivalence class or Equivalence Partition is the act of dividing the given input set by a relation into groups or classes which is treated the same by the module or which should produce the same result. Any data value within a class is equivalent in terms of testing, to any other value.

Equivalence partitioning is a software testing design technique that involves identifying a small set of representative input values that produce as many different output conditions as possible. One representative sample from each partition is picked up for testing since all the values produce equal and same output. This technique tries to define test case that uncovers classes of errors, thereby reducing the total number of test cases that must be developed.

The step for using equivalence class testing are:

1. Identify the Equivalence class from the given requirements or scenario.
2. Create a test case for each equivalence class.
3. From the results obtained for one member in an equivalence class extrapolate results for all values in that partition.

Examples:
The electricity bill computed by the service provider has a fixed component as well as a running component. All customers are charged at a rate of $40 flat as a fixed component. In addition to this, they would be changed a running component and/ or a fine, depending upon their amount of consumption or usage.

The rules for this are given below:

• If the number of units consumed by the consumer is less than 10 units, then the running cost is not charged for the consumer.
• If the nimber of units is between 11 and 20, then the running cost is charged at $1 per unit.
• If the number of units is between 21 and 40, then the running cost is charged at $2 per unit.
• If the number of units exceeds 40, then the running cost is charged at $5 per unit.
• Use equivalence class partitioning to decide on the test cases to be designed.

Test Design Techniques Black box testing

What is Black box Testing ?
Black box testing is a strategy in which testing is based solely on the requirements and specifications. Here, testing the software is done based on output requirements and without any knowledge of the internal structure or coding in the program.

Process involved in Black box testing:
First, the requirements or specifications are analyzed and valid inputs are chosen based on the specification. Then we need to determine the expected outputs for those inputs. Now tests are constructed with the selected inputs to execute the test. After executing the tests actual outputs are compared with expected results.
Note: Test design techniques are used to design test cases so that an effective set of test cases can be prepared.

What are the Test design techniques for black box testing:
Following are the black box testing techniques.

• Equivalence class partitioning
• Boundary value Analysis
• Decision Tables
• State transition Diagrams
• Orthogonal Arrays
• All Pairs

Recommended:
Equivalence class testing technique examples
Boundary value analysis with examples
Decision Table Testing Example
state transition testing sample
How To do Testing with orthogonal Array 

Why Test cases must be designed ?

Need for Test Design:
Just like code is designed and developed, test cases also must be designed and then written. Since test design is required to derive at an optimal test suite which is of reasonable size and uncovers as many errors as possible. Also have to consider that randomly selecting or writing test cases does not provide effective testing.

Let's see an example of simple programming error.
If (X > Y) max = X;
Else max  = X;

Here, for the above program if we developed large test suite without design such as {(X=3, Y=2), (X=10, Y=1), (X=100, Y=0),(X=5,  Y=4) cannot going to detect any error. But if we have a design as that of small test suite {(X=3, Y=2), (X=5, Y=6)} can easily detect the error.

Requirements Traceability And Test Cases

What is Requirements Traceability ?
Functional or software requirement specification  is the document which describes stakeholder's requirements which should be traced with deliverables produced during the software development life cycle.

What are the objectives of Requirements Traceeability ?
The objectives of requirement traceability, in general, are to

• Ensure that the deliverables meet the requirement specification.
• Ensure that no requirements are missed and therefore  that all are delivered.
• Identify which deliverables implement which requirements.
• Identify any deliverables that cannot be tracked back to a requirement. This may be either due to undocumented requirements or to the unnecessary introduction of features that are not actually required by the stakeholders.

Requirements Traceability and Test cases:
Every requirement should be traced to the individual test cases. This is required to ensure that all the requirements have been covered in the test cases and also to identify what tests are necessary to verify a particular requirement has been implemented successfully.

How to write a test case from use case

Use Case to Test Case Conversion:
Let us see the process for generating test cases from a fully detailed use case. The first step is to for each use case, generate a full set of high level test scenarios. For each high level scenario, identify at least one positive or negative test case and the conditions that will make it "execute". Then for each test case, identify the data values with which to test. At last document the test case as a combination of stepps and expected results including the above identified data values.

Tester should consider following concepts under preparation of test case:

• Ensure each section in use case has one or many test cases.
• Ensure integrity between modules are maintained throughout the test cases.
• Need to ensure inter dependencies external to the system are taken care in test cases.

What is Test Case? Why Do We Need

What is a Test case?
IEEE Standard 610 (1990) defines a test case as follows:
A set of inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement."

A test is not only designed to reveal a defect but also to learn something from the defect.

Why Do We a Test Case?

• Test cases identify and communicate the conditions that will be implement in test.
• Necessary to very successful and acceptable implementation of the product requirements.
• Help finding the problems in the requirements or design of an application.
• To determine whether we meet the client expectations.
• Help managers make ship / no-ship decisions.

What would happen if there is no Test Case?
If there is no test case then we cannot test effectively and so poor quality  of the deliverables to the client. Other thing is no requirements coverage tracking and take more time for execution.