Index of /~billard/se/cs4320/ex4

 Name                    Last modified       Size  Description

 Parent Directory        20-Apr-2008 08:49      -  
 avg                     19-May-2009 09:03    23k  
 avg.bat                 19-May-2009 09:03     1k  
 avg.c                   19-May-2009 09:03     5k  
 comp.bat                19-May-2009 09:03     1k  
 max                     19-May-2009 09:03    23k  
 max.c                   19-May-2009 09:03     1k  
 maxab                   19-May-2009 09:03    23k  
 maxab.bat               19-May-2009 09:03     1k  
 maxab.c                 19-May-2009 09:03     4k  
 maxabc                  19-May-2009 09:03    23k  
 maxabc.bat              19-May-2009 09:03     1k  
 maxabc.c                19-May-2009 09:03     2k  
 minmax                  19-May-2009 09:03    23k  
 minmax.c                19-May-2009 09:03     1k  
 passfail                19-May-2009 09:03    23k  
 passfail.bat            19-May-2009 09:03     1k  
 passfail.c              19-May-2009 09:03     3k  
 print.bat               19-May-2009 09:03     1k  
 printdoc.bat            19-May-2009 09:03     1k  
 while1                  19-May-2009 09:03     7k  
 while1.c                19-May-2009 09:03     1k

CS 4320 Software Testing and QA: Ex4 WHITE BOX TEST
===================================================

Platform: UNIX or PC

Given: source code

Goal: perform white box testing using cyclometric complexity

White Box Test: given source code, design test cases that exercise various parts of the code.

The idea is that the more paths that are "covered" by the test cases, the more confident
we are in the software - as long as the test cases yield correct answers. But how many
test cases should we make? A million? Two million? The CYCLOMETRIC COMPLEXITY tells us
exactly how many cases we need.

In Ex3, you first learned about Cyclometric Complexity:

Each predicate in an IF or WHILE condition creates a unique path.
The number of independent paths is the Cyclometric Complexity and is equal to:

Predicates + 1 = Edges - Nodes + 1 = Regions - 1

where Edges, Nodes, Regions are determined by drawing a FLOWGRAPH of the code.

Path Coverage for white box strategy tests each independent path based on the
cyclometric complexity.

The example from Ex3:

CODE:

if (a)
if (b)
x;
else
y;
else
y;
z;

FLOWGRAPH:
..............
. .
F . T .
+----a----+ .
R4 | R1 | R3 .
v F v .
y<--------b .
| R2 | T .
| v .
| x .
| | .
+--->z<---+ .
. .
..............

CYCLOMETRIC COMPLEXITY:

CC = P + 1 = E - N + 1 = R - 1

CC = 2 + 1 = 6 - 4 + 1 = 4 - 1 = 3

TEST CASES:

a b
------------
True True
True False
False *

In summary, the code yields a flow graph with a particular complexity.
This complexity tells us how many test cases we expect to exactly
"cover" all of the code. That is what you will do in this exercise.

Metrics:

The cyclometric complexity - which counts the number of predicates - is
an example of a "metric" to evaluate the complexity of source code. As the
cyclometric complexity increases, so do the number of faults. It is suggested
that a cyclometric complexity greater than 10 for a function should be avoided.
Also, the number of faults increases with the metric Lines of Code (LOC or KLOC).

In the following examples, the key points will be:

CODE: The starting point will be given C functions which are designed to compute something
specific.

FLOWGRAPHS: Using the source code, draw the corresponding flowgraph, where the main
purpose is to split at branch points (predicates).

CYCLOMETRIC COMPLEXITY: Compute CC = P + 1 = E - N + 1 = R - 1

PATH COVERAGE: The CC will be the exact number of test cases needed to provide 100%
path coverage.

INDEPENDENT PATHS: Determine each of the paths through the code, and there should be
the same number as CC. No more, no less. What does "independent" mean? Remember
Math 2101 Linear Algebra? You formed simultaneous equations which were added or
subtracted with each other. Our independent paths will NOT be the addition or
subtraction of another path. More on this later.

BASIS CYCLES: Before we get to loops, basis cycles will look like paths with this
exception: paths start at the top of the function and work their way to the bottom.
A basis cycle does the same thing but returns to the top (just like the artificial
edge). After we introduce loops, basis cycles will not look like paths at all,
they will just be the loop itself. This is all a mathematical concept that we
won't explore any further - but you will be responsible for computing the cycles.
The key point will be that the number of BASIS CYCLES will be the same as the
CYCLOMETRIC COMPLEXITY, which is the same as the number of INDEPENDENT PATHS.

INSTRUMENTATION: You probably call this "debug". You'll put debug code in the given
C function which highlights which "direction" the logic is flowing. That is, it will
print the PATH the logic is following. This will help you see if your testing is
working.

TEST CASES: Now that you know the CC number of PATHS, and have INSTRUMENTATION code to
print the paths, design the actual test cases which force the function along a particular
path. Each test case specifies the input to the function, and the expected return value.
Although CC determines the exact number of needed tests cases for 100% path coverage, there
may be infeasible paths. That is, the flowgraph has a mathematical path but that path
may be contradictory in terms of actual data in the code. For example, a TRUE on an IF
statement may make it infeasible for a nested IF statement to ever go the FALSE branch.

AUTOMATED TEST: Given the TEST CASES cases on paper, you will create Unix batch scripts to
execute the function CC number of times, and feed the function a new test case each time.
The INSTRUMENTATION will show the logic following the INDEPENDENT PATHS.

You may want to look at the examples provided on the directory now, but here is a summary
of paths and cycles that you can refer to:

What are the strategies for creating Independent Paths and their significance?

The flowgraphs - from Ex3 - present the logic of the code. Graph theory
tells us that the cyclometric complexity (predicates + 1) will determine
the number of independent paths through the code. "Independent" means
one path is not some combination of addition/subtraction of other paths.
Although testing cannot prove the absence of bugs, we feel more confident
about the software if we can test all possible flows through the code.
And know the number of such paths (CC) helps us to construct the
appropriate number of test cases.

Now that we know the number of such paths, we can create a path by
starting at the top of the flowgraph, flow towards the bottom, but be
sure to touch at least one new edge. Be patient, not greedy. Try to avoid
consuming lots of edges in one path. There will always be a way to list
CC number of paths and to touch at least edge in each one. If you come
up with CC number of paths, and each one touches at least one new path,
you know you have the correct solution.

What are the strategies for creating a Basis Set of Cycles?

There will always be CC number of Basis Cycles, just like Independent
Paths. And "Basis" has the same meaning as "Independent": not some
combination of addition/subtraction of other cycles.

Without loops, the cycles look exactly like the paths but they complete
a "cycle" by returning to the starting node, using the artificial
edge. With loops, some cycles don't look anything like a path. By
definition, a cycle must start at a node, and return to the node, but
cannot visit any other node more than once. So in the case of a loop,
a cycle will start/stop at the node representing the while condition.
Other cycles will start at the top but "bounce" off this loop with a
false condition. NO CYCLE will ever start at the top and go around the
loop too. For example, consider:

a;
while (b)
x;
y;

Two cycles:
a,b,y,a (bounce off loop, and use artificial edge)
b,x,b (just the loop, touch a new edge)

But this is NOT a cycle:
a,b,x,b,y,a

Why? Because "b" appears twice and it isn't the start/stop node.

Just like "Independent" paths, if you touch a new edge, you can be sure
that it's a "Basis" cycle.

Note that the paths would have been:

a,b,y (don't be greedy)
a,b,x,b,y (touch a new edge)

What is the motivation for Basis Cycles?

For this course, the only motivation is to test your logic skills as a
computer science major. Note that CS 2150 Discrete Math is a prerequisite
and covers Graph Theory. It probably covered the concept of a cycle but
certainly not "basis" cycles.

REMINDERS:

1. If you touch a new edge then it is INDEPENDENT.

2. Let's say (as in Ex3.3 #9) Path4 = Path1 + Path2 - Path3.

But that means that: Path3 = Path1 + Path2 - Path4 (etc.)

So you can pick ANY 3 out of the 4. Although there are 4 PATHS,
only 3 (but any 3) can be considered INDEPENDENT.

It is recommended to list the 3 in such an order that a new edge
is touched - just a guarantee.

EXAMPLES:

The first example is simple but illustrates all of the above so be sure
to study it in detail.

a. % more maxab.c
% more maxab.bat
% maxab.bat

b. % more passfail.c
% more passfail.bat
% passfail.bat

c. % more maxabc.c
% more maxabc.bat
% maxabc.bat

d. % more avg.c
% more avg.bat
% avg.bat

Perform Path Coverage White Box Test:

e. % more minmax.c

in the source minmax.c:

0. label the `lines' of code with L(1), L(2), etc.
1. FLOWGRAPH (or use a word processor)
2. CYCLOMETRIC COMPLEXITY
3. BASIS SET OF CYCLES
4. LINEARLY INDEPENDENT PATHS 5. FEASABLE (YES/NO) 6. TEST CASES 7. RESULT

TRY TO GET AS MANY FEASABLE INDEPENDENT PATHS AS POSSIBLE.

% gcc -o minmax minmax.c

create minmax.bat which includes all of your test cases.

minmax.bat should also indicate expected results like the other .bat files.

% minmax.bat >minmax.out // instructor needs to see these too

f. % more max.c

repeat the above.

% gcc -o max max.c
% max.bat >max.out

g. % more while1.c

repeat the above.

% gcc -o while1 while1.c
% while1.bat >while1.out

g. project notebook (see print.bat):

README
maxab.c
maxabc.c
avg.c
passfail.c
minmax.c
minmax.bat
minmax.out
max.c
max.bat
max.out
while1.c
while1.bat
while1.out