Multiple Condition Coverage

Multiple Condition Coverage is also known as Condition Combination Coverage.

In Multiple Condition Coverage for each decision all the combinations of conditions should be evaluated.

Let's take an example:

    if (A||B)
    then
    print C

Here we have 2 Boolean expressions A and B, so the test set for Multiple Condition Coverage will be:

    TEST CASE1: A=TRUE, B=TRUE
    TEST CASE2: A=TRUE, B=FALSE
    TEST CASE3: A=FALSE, B=TRUE
    TEST CASE4: A=FALSE, B=FALSE

As you can see that there are 4 test cases for 2 conditions. Similarly there will be 8 test cases for 3 conditions.

So you can say that if there are n conditions, there will be 2^n tests.

Multiple Condition Decision Coverage

Multiple Condition Decision Coverage(MCDC) is also known as Modified Condition Decision Coverage.

In MCDC each condition should be evaluated at least once which affects the decision outcome independently.

Example for MCDC

if {(X or Y) and Z} then

To satisfy condition coverage, each Boolean expression X,Y and Z in above statement should be evaluated to TRUE and FALSE at least one time.

The TEST CASES for condition coverage will be:

    TEST CASE1: X=TRUE, Y=TRUE, Z=TRUE
    TEST CASE2: X=FALSE, Y=FALSE, Z=FALSE

To satisfy the decision coverage we need to ensure that the IF statement evaluates to TRUE and FALSE at least once. So the test set will be:

    TEST CASE1: X=TRUE, Y=TRUE, Z=TRUE
    TEST CASE2: X=FALSE, Y=FALSE, Z=FALSE

However for MCDC the above test cases are not sufficient because in MCDC each Boolean variable should be evaluated to TRUE and FALSE at least once and also affect the decision outcome.

So to ensure MCDC we need 4 more test cases.

    TEST CASE3: X=FALSE, Y=FALSE, Z=TRUE
    TEST CASE4: X=FALSE, Y=TRUE, Z=TRUE
    TEST CASE5: X=FALSE, Y=TRUE, Z=FALSE
    TEST CASE6: X=TRUE, Y=FALSE, Z=TRUE

    In test case 3 decision outcome is FALSE
    In test case 4 decision outcome is TRUE
    In test case 5 decision outcome is FALSE
    In test case 6 decision outcome is TRUE

So in the above test cases you can see that the change in the value of Boolean variables made a change in decision outcomes. 

Condition Coverage Or Predicate Coverage

Condition coverage is also known as Predicate Coverage
Condition coverage is seen for Boolean expression, condition coverage ensures whether all the Boolean expressions have been evaluated to both TRUE and FALSE.

Let us take an example to explain Condition Coverage

    IF ("X && Y")

In order to suffice valid condition coverage for this pseudo-code following tests will be sufficient.

    TEST 1: X=TRUE, Y=FALSE
    TEST 2: X=FALSE, Y=TRUE

Note: 100% condition coverage does not guarantee 100% decision coverage.

Decision Coverage Or Branch Coverage

Decision Coverage is also known as Branch Coverage.

Whenever there are two or more possible exits from the statement like an IF statement, a DO-WHILE or a CASE statement it is known as decision because in all these statements there are two outcomes, either TRUE or FALSE.

With the loop control statement like DO-WHILE or IF statement the outcome is either TRUE or FALSE and decision coverage ensures that each outcome (i.e. TRUE and FALSE) of control statement has been executed at least once.

Alternatively you can say that control statement IF has been evaluated both to TRUE and FALSE.

The formula to calculate decision coverage is:
Decision Coverage=DC
DC=(Number of decision outcomes executed/Total number of decision outcomes)*100%

Research in the industries have shown that even if through functional testing has been done it only achieves 40% to 60% decision coverage.

Decision coverage is stronger that statement coverage and it requires more test cases to achieve 100% decision coverage.

Let us take one example to explain decision coverage:

    READ X
    READ Y
    IF "X > Y"
    PRINT X is greater that Y
    ENDIF



To get 100% statement coverage only one test case is sufficient for this pseudo-code.

    TEST CASE 1: X=10 Y=5


However this test case won't give you 100% decision coverage as the FALSE condition of the IF statement is not exercised.

In order to achieve 100% decision coverage we need to exercise the FALSE condition of the IF statement which will be covered when X is less than Y.
So the final TEST SET for 100% decision coverage will be:

    TEST CASE 1: X=10, Y=5
    TEST CASE 2: X=2, Y=10


Note: 100% decision coverage guarantees 100% statement coverage but 100% statement coverage does not guarantee 100% decision coverage.

Statement Coverage Or Line Coverage

Statement coverage is also known as line coverage.
The formula to calculate statement coverage is:

Statement Coverage=SC
SC=(Number of statements exercised/Total number of statements)*100

Studies in the software industry have shown that black-box testing may actually achieve only 60% to 75% statement coverage, this leaves around 25% to 40% of the statements untested.

To illustrate the principles of code coverage let's take one pseudo-code which is not specific to any programming language. We have numbered the code lines just to illustrate the statement coverage example however this may not always be correct.

    READ X
    READ Y
    IF X>Y
    PRINT “X is greater than Y”
    ENDIF

Let us see how can we achieve 100% code coverage for this pseudo-code, we can have 100% coverage by just one test set in which variable X is always greater than variable Y.

    TEST SET 1: X=10, Y=5

A statement may be a single line or it may be spread over several lines. A statement can also contain more than one statement. Some code coverage tools group statements that are always executed together in a block and consider them as one statement.

White Box Testing

White-box testing is concerned with testing, the implementation of the program. The intent of this testing is not to exercise all the different input or output conditions but to exercise the different programming structures and data structures used in the program.

White-box testing is also called as Structural Testing or Glass box testing. It is a good practice to perform white box testing during the unit testing phase.

Black Box Testing

Black-box testing enables the software engineer to derive sets of input conditions that will fully exercise all functional requirements for a program. It is a complementary approach that is to uncover the errors. Black-box testing attempts to find the errors in, incorrect missing functions, interface errors, errors in data structures, external data base access, performance error or termination errors.

Black-Box Testing is also called as Behavioral testing. Black-box testing is usually described as focusing on testing functional requirements.

Testing Web Applications

Web Application can have thousands of users than a conventional, non-Web Application. Web Application checklist is as follows:-

    Functionality Testing.
    Usability testing.
    Interface testing.
    Compatibility testing.
    Performance testing.
    Security testing.

Test all the Links in web pages.

Test the particulars like:-

    Test the outgoing links from all the pages.
    Test all internal links.
    Test links on the same pages.
    Test links used to send the email to admin or other users from web pages.
    Test to check if there are any orphan pages.
    Test the Anchor text links.

Test Database:-
 
Data consistency is very important in web application.
 
Check for data integrity.
    Errors while doing Database functionality like Editing, Deleting, Inserting, Modifying the forms.

Test the Cookies:-

    Test the application by enabling or disabling the cookies.
    Test the session cookies check for login sessions and user status after session ends.
    Check effect on application security by deleting the cookies.

Test the Content:-

    Content should be logical and easy to understand.
    Check for spelling errors.
    Use of dark colors annoys for font, frames etc...
    All the anchor text links should be working properly.
    Images should be placed properly with proper sizes.

Test Interfaces:-

    Check if all the interactions between these servers are executed properly.
    Errors are handled properly.
    Check what happens if user interrupts any transaction in-between?
    Check what happens if connection to web server is reset in between?

Test application performance:-
    Load Testing.
    Stress Testing.

Test for Security:-

    Test by pasting internal URL directly into browser address bar.
    Try directly changing the URL site ID parameter to different site ID which is not related to log in user.
    Try some invalid inputs in input fields.

Equivalence Partitioning

Equivalence partitioning (EP) is a black box testing technique. This technique is very common and mostly used by all the testers informally. Equivalence partitions are also known as equivalence class partitioning.

As the name suggests Equivalence partitioning is to divide or to partition a set of test conditions into sets or valid and invalid groups that can be considered same while testing.

For example if you have to test a condition say Age field should only accept a number ranging from 0 and 100. You can't test for each value. As you all know that exhaustive testing of the software is not feasible task. In such cases equivalence partitioning technique is applied. Here the test data would be divided into partitions of valid and invalid range. In our case, valid range would be 0 to 100 and invalid range would be >100 and <0. While testing a data from each class or partition is considered and tested.

State Transition Graphs

State transition testing is used for systems where some aspect of the software system can be described in 'finite state machine'. This means that the system can be in a finite number of different states, and the transitions from one state to another are determined by the rules of the 'machine'.

What is a finite State System?

Any system where you get a different output for the same input, depending on what has happened before is a finite state system. A finite state system is often shown as a state diagram.

Let us take an example to explain this in detail:

Suppose you want to withdraw $500 from a bank ATM, you may be given cash. After some time you again try to withdraw $500 but you may be refused the money (because your account balance is insufficient). This refusal is because your bank account state has changed from having sufficient funds to cover withdrawal to having insufficient funds. The transaction that caused your account to change its state was probably the earlier withdrawal. A state diagram can represent a model from the point of view of the system or the customer.

A state transition model has four basic parts:-

    1.The states that the software may occupy (funded/insufficient funds)
    2.The transitions from one state to another (all transitions are not allowed)
    3.The events that cause a transition (like withdrawing money)
    4.The actions that result from a transition (an error message or being given your cash)

Please note that in any given state, one event can cause only one action, but that the same event from a different state may cause a different action and a different end state.

Decision Table Testing

What is a Decision Table?

It is a table which shows different combination inputs with their associated outputs, this is also known as cause effect table.

In EP and BVA we have seen that these techniques can be applied to only specific conditions or inputs however if we have different inputs which result in different actions being taken or in other words we have a business rule to test where there are different combination of inputs which result in different actions.

For testing such rules or logic decision table testing is used.

It is a black box test design technique.

All Pair Technique

"All Pairs" is a test design technique that can be used when there are several variables to test. The test cases include all the pairs of values for every variable. The output of "All Pairs" technique is that every value is paired with another value at least once.

Here are some examples of testing several variables:

-    A search screen with several search criteria. 
-    Configuration options of software. 
-    Configuration options of hardware. 
-    Compatibility testing across different OS and browser combinations.

"All Pairs" drastically reduces the number of test cases required to test software and it can help you find most errors with the smallest number of test cases.

Example:

Let say, for example, there are 3 parameters that can be given for a search criteria: P1, P2 and P3.

-    P1 can have the values as '10' and '50'.

-    P2 can have the values as 'g' and 'c'.

-    P3 can have the values as 'a' and 'b'.

Step 1 - calculate all combinations

The Cartesian product will be:

P1 X P2 X P3 = {10, 50} X {g, c} X {a, b}

In tabular format:

Test Case #	Values Participates	Result
1	{10, 50}X{g, c}X{a, b}	(10, g, a)
2	{10, 50}X{g, c}X{a, b}	(10, g, b)
3	{10, 50}X{g, c}X{a, b}	(10, c, a)
4	{10, 50}X{g, c}X{a, b}	(10, c, b)
5	{10, 50}X{g, c}X{a, b}	(50, g, a)
6	{10, 50}X{g, c}X{a, b}	(50, g, b)
7	{10, 50}X{g, c}X{a, b}	(50, c, a)
8	{10, 50}X{g, c}X{a, b}	(50, c, b)

Step 2 – base table

Now, let's create a table that each column represents a variable and each row represents a test case combination:

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
3	10	c	a
4	10	c	b
5	50	g	a
6	50	g	b
7	50	c	a
8	50	c	b

Step 3 - pick the last row (mark in red):

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
3	10	c	a
4	10	c	b
5	50	g	a
6	50	g	b
7	50	c	a
8	50	c	b

Step 4 – If all pairs exist in other rows – delete the last row and go up, if not keep the row and go up to the next row:

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
3	10	c	a
4	10	c	b
5	50	g	a
6	50	g	b
7	50	c	a
8	50	c	b

We can see that the pair (50, c) exists in test case 7, the pair (50, b) exists in test case 6 and the pair (t, b) exists in test case 4. Because all pairs in test case 8 (the last row) exists in above rows, we can delete test case 8.

The new table will be:

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
3	10	c	a
4	10	c	b
5	50	g	a
6	50	g	b
7	50	c	a

Now, let's move one row up to test case 7:

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
3	10	c	a
4	10	c	b
5	50	g	a
6	50	g	b
7	50	c	a

We don’t have the couple (50, b) so we keep the line (test case 6) and move up to the next row up to test case 5.

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
3	10	c	a
4	10	c	b
5	50	g	a
6	50	g	b
7	50	c	a

We can see that the pair (50, g) exists in test case 6, the pair (50, a) exists in test case 7 and the pair
(g, a) exists in test case 1. Because all pairs in test case 5 (the current row) exists in other rows, we can delete test case 5.


The new table will be:

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
3	10	c	a
4	10	c	b
6	50	g	b
7	50	c	a

Now, let's move one row up to test case 4.

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
3	10	c	a
4	10	c	b
6	50	g	b
7	50	c	a

We don’t have the couple (c, b) so we keep the line (test case 4) and move up to the next row up to test case 3:

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
3	10	c	a
4	10	c	b
6	50	g	b
7	50	c	a

We can see that the pair (10, a) exists in test case 1, the pair (10, c) exists in test case 4 and the pair (c, a) exists in test case 7. Because all pairs in test case 3 (the current row) exists in other rows, we can delete test case 3.


The new table will be:

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
4	10	c	b
6	50	g	b
7	50	c	a

Now, let's move one row up to test case 2:

Test Case #	P1	P2	P3
1	10	g	a
2	10	g	b
4	10	c	b
6	50	g	b
7	50	c	a

We can see that the pair (10, g) exists in test case 1, the pair (10, b) exists in test case 4 and the pair (g, b) exists in test case 6. Because all pairs in test case 2 (the current row) exists in other rows, we can delete test case 2.

The new table will be:

Test Case #	P1	P2	P3
1	10	g	a
4	10	c	b
6	50	g	b
7	50	c	a

Now, let's move one row up to test case 1.

Test Case #	P1	P2	P3
1	10	g	a
4	10	c	b
6	50	g	b
7	50	c	a

We don’t have the couples (10, g), (10, b), (g, b) so we keep the line.

Test Case #	P1	P2	P3
1	10	g	a
4	10	c	b
6	50	g	b
7	50	c	a

Now, no more lines left which means that after using "All Pairs" technique, we have 4 test cases to test. We reduced the number of test cases from 8 to 4 (50% less).

Error Guessing

What is Error Guessing?

This is a Test design technique where the experience of a tester is used to find the components of software where defects might be present.

It is mostly done by experienced tester's who can use their past experience to find defects in software.

Error guessing has no rules for testing, it only uses the tester's previous skills.

In error guessing tester's can think of situations where software will fail.
 
For example:

    1. Division by zero.
    2. Pressing submit button on form without filling any entries.
    3. Entering wring data in the fields and checking software behavior.

Use Case Technique

Before explaining Use case Testing lets first understand what is a Use Case?

1. A Use case is a description of a particular use of the system by the end user of the system.

2. Use cases are a sequence of steps that describe the interactions between the user and the software system.

3. Each use case describes the interactions the end user has with the software system in order to achieve a specific task.

What is Use case Testing?

Use case testing is a technique that helps us identify test cases that exercise the whole system on a transaction by transaction basis from start to finish.

Use cases are defined in terms of the end user and not the system, use case describe what the user does and what the user sees rather than what inputs the software system expects and what the system outputs.

Use cases use the business language rather than technical terms.

Each use case must specify any preconditions that need to be met for the use case to work. Use cases must also specify post conditions that are observable results and a description of the final state of the system after the use case has been executed successfully.

Boundary Value Analysis

A boundary value is a value on the edge of an equivalence partition. Boundary value analysis is all about testing the component at the boundaries.
For example: Consider the same example that is considered for equivalence partitioning.

There is an Age field that should accept a range of values from 0 to 100 only. For this the boundaries would be 0 and 100. Testing at the boundary would yield 3 data values at each end that should be used for testing. So for 0 the boundary the values would be -1, 0 and 1 (It is to have a value less than the boundary value, the boundary value and the next value) and the values for next boundary 100 are 99, 100 and 101

All the 6 values have to be used to test Age field.

Test closure

After completion of all the test execution cycles and user acceptance test, the test lead prepares the final test summary report containing all the deliverables.

Defect reporting

After the test execution is done, the testing team prepares a defect summary sheet.

What is a defect?

In general with specific to a project, anything that varies from the actual requirement/functionality is a defect.

Defect Reporting Format

As per the IEEE 829 format a defect report must contain the following

a. Defect Id
b. Description
c. Module
d. Test case name
e. Reproducible
f. Status
g. Severity
h. Priority
i. Reported by
j. Reported on
k. Assign to
l. Build number
m. Fixed by
n. Resolved by
o. Resolved on
p. Resolution type
q. Approved by

Test execution

Once the test case authoring is done and reviewed thoroughly, testing team concentrates on build release from development side/client and test execution on that build.

Test Execution levels:

Once the testing team receives a new build different levels of executions are done.

Level 1 (Smoke Test/Build Verification test/Tester acceptance test/sanity test:

After receiving the initial build, test engineers verify if the basic functionality of the build to estimate the stability for complete testing. In smoke testing only a few test cases from the actual test case suite are selected for execution. If the test engineers find that the build is unstable they reject the build for testing.

Level 2(comprehensive testing)

Once the smoke testing is done and the build is stable, comprehensive testing is done on the build. In this one cycle of execution covering all the test cases is performed. Defects log report is prepared after the execution and sent to development team for fix.

Level 3(Regression Test)

After comprehensive testing is done the initial build and defects are reported. The development team fixes the defects and sends a new build (with defects fixed) to the testing team. The testing team performs regression testing on the new build to verify if the defects are fixes and if any new defects are found because of the previous fixed defects.

Test design

Test design involves authoring of test cases. What is a Test case?

IEEE Standard 610 (1990) defines test case as follows:

“(1) A set of test inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.

“(2) (IEEE Std 829-1983) Documentation specifying inputs, predicted results, and a set of execution conditions for a test item.” Test Case Design Techniques

Black box Design Techniques:

a. Equivalence Class Partitioning
b. Boundary Value Analysis
c. Decision Table Testing
d. State Transition Testing
e. Use Case Testing
f. Error Guessing Technique

White Box Design Techniques:

a. Control Flow Testing
b. Data Flow Testing
c. Decision Coverage
d. Statement coverage
 
Key Points to be followed in Designing

A Test case should
    a. Start with the word verify or check
    b. Contain a maximum of 15 steps
    c. Specify what the tester has to perform
    d. Give consistent results irrespective of testers
    e. Be divided into positive and negative scenarios
    f. Has a name and number
    g. Should be reusable
    h. Should be simple and easy to understand
 
Test Case Format

As per the IEEE format a test case should contain the following
a. Test case Id
b. Requirement ID
c. Test Case priority
d. Test Case Authoring Status
e. Authoring Date
f. Test Case Title
g. Pre-Condition
h. Test Data
i. Test Steps
j. Expected Result
k. Actual result
l. Execution status
m. Defect Id
n. Defect Severity
o. Execution Date
In general it can be as follows
Ex: A Test Case to open Note pad

Step no	Steps to perform	Expected Result	Actual Result
1	Click on ‘Start’ Menu on the Left hand bottom of the screen	A window containing ‘all programs’ is displayed	Same as Expected
2	Click on ‘Run’	A run window is opened with label ‘open’	Run window is successfully opened
3	Enter ‘Notepad’ in the text box and click on ok button	Notepad is opened with File, Edit, Format, View and help menu	Note pad is successfully opened