4.4 Experimental Results
4.4.2 Case Studies for MIPS Processor and Leon3 Processor
A set of macros corresponding to the instructions of the MIPS processor are used for developing the constituents of the instruction library. For the MIPS processor, the enhancement in the development of the test synthesis using the proposed RSBST scheme is shown in Fig. 4.5(a). The average fault coverage of the conventional µGP scheme [2]
with behavioral fault model achieves an adequate coverage (80-85%) after 50 generations, whereas the greedy-based GA proposed in the last chapter could only cover 85% of the faults after 75 generations. Eventually, 93.9% of the behavioral faults are detected by the
10 20 30 40 50 60 70 80 90 100
0 50 100 150 200 250 300 350 400
Fault Coverage(%)
Generations
Existing microGP Greedy-based GA Proposed RSBST
a) Average Coverage for MIPS Processor
10 20 30 40 50 60 70 80 90 100
0 50 100 150 200 250 300 350 400
Fault Coverage(%)
Generations
Existing microGP Greedy-based GA Proposed RSBST
b) Average Coverage for Leon3 Processor
Figure 4.5: Average Fault Coverage for MIPS Processor and for Leon3 Processor over 400 Generations using 1) µGP [2] with Behavioral Fault Model 2) Greedy-based GA proposed in the last chapter 3) Proposed RSBST
µGP approach [2] and 96.3% of faults are detected by the greedy-based GA. However, our RSBST code synthesis yields more than 85% of fault coverage before 50 generations and conclusively, carries out an adequate coverage of 96.1%.
For the Leon3 processor, the progress in the achieved fault coverage using the RSBST scheme is shown in Fig. 4.5(b). The conventionalµGP approach [2] with the behavioral fault model, yields a fault coverage of 80-85% before 150 generations whereas the greedy- based GA accomplishes above 80% coverage only after 200 generations. Finally, µGP approach [2] could detect 92.9% of the faults and the greedy-based GA comes up with a fault coverage of 95.8%. However, the proposed RSBST code synthesis covers more than 85% of the possible faults before 100 generations and ends with a final fault coverage of 95.5%.
The test synthesis for MIPS processor was conducted module-by-module whereas monolithic test programs were generated for the Leon3 processor. The processor model describes the RTL model of the processor to be tested in the hardware description language VHDL, either in synthesizable or simulatable form. Here, the RTL model is subjected to module partitioning which is realized by breaking down the RTL design into several functional units and testing them separately. So, each processor module corresponds to a single hardware block and therefore, there are as many modules as the number of valid digital blocks in the processor model.
The coverage and test synthesis time for the five major modules of the MIPS processor
4.4 Experimental Results
Table 4.3: Achieved Coverage and Synthesis Time of MIPS Processor Modules MIPS
Processor Module
Conventional µGP by G.Squillero [2]
Greedy GA proposed
in the last chapter Proposed RSBST Coverage
(%))
Synthesis Time (Hours)
Coverage (%))
Synthesis Time (Hours)
Coverage (%))
Synthesis Time (Hours)
ALU 100.00 24.50 100.00 33.73 100.00 18.07
PC 100.00 11.50 100.00 15.83 100.00 8.48
RF 96.67 15.00 96.67 20.65 96.67 11.06
ALU Control 90.24 21.50 94.87 29.63 94.27 15.86
Control Unit 83.83 49.50 90.32 68.16 90.12 36.53
Total 93.90 122.00 Hrs 96.30 168.00 Hrs 96.10 90 Hrs
is shown in Table 4.3. The overall test set constitute the test programs synthesized for the validation of each module. In the conventionalµGP [2], achieved coverage (83.83%) was inefficient for the control unit module but the synthesis was reasonably fast (49.5 hours). So, the coverage of the control unit was improved towards 90.32% for the greedy coverage method, which encounters a longer test synthesis of 68.16 hours.
Some of the behavioral faults of the control unit could only be tested using rare se- quences of instructions only. So, we make use of a larger solution space for the population of test individuals. Since the RSBST code synthesis is faster, this larger population of test individuals helps in developing instruction sequences that could detect harder be- havioral faults for the control unit. As a result, test programs with coverage above 90%
is synthesized for the control unit within 36.53 hrs using the proposed RSBST technique.
For the remaining modules, minimum coverage of 94% is guaranteed with the overall evolutionary test synthesis terminates in 90 hours. Now, the amount of simulation re- sponses reused for the chromosomes, bypassing the fault simulation, is discussed in the next subsection.
However, the fault coverage for ALU control and control unit achieved by RSBST technique is slightly smaller than that of Greedy GA method. This reduction is due to the high-level logic simulation of test programs in RSBST approach. In RSBST approach, we conduct a high-level logic simulation for a new offspring test program to check its reusability. If its logic simulation results are matching with that of any of its parents, the actual simulation results of parent test program are reused for the offspring test program. In those cases, actual fault simulation of the offspring is not conducted.
Table 4.4: MIPS Processor - Achieved Coverage and Time of the 1) µGP [2] with Behavioral Fault Model 2) Greedy-based GA proposed in the last chapter 3) Proposed RSBST Method
Framework Simulation Environment
Behavioral Fault Coverage
Test Synthesis Time
Chromosome
Reuse Remarks Conventional
µGP by G.Squillero [2]
Modelsim
Version 5.7a 93.9% 122 Hours 66.6%
Lesser fault coverage but test synthesis is faster.
Greedy GA proposed in the last chapter
GHDL 96.3% 168 Hours 66.6%
Improved fault coverage but test synthesis consumes huge time Proposed
RSBST
Modelsim
Version 10.5b 96.1% 90 Hours 82.1%
Adequate fault coverage and faster test synthesis
Table 4.5: Leon3 Processor - Achieved coverage and Time of the 1)µGP [2] with Behavioral Fault Model 2) Greedy-based GA proposed in the last chapter 3) Proposed RSBST Method
Framework Simulation Environment
Behavioral Fault Coverage
Test Synthesis Time
Chromosome
Reuse Remarks Conventional
µGP by G.Squillero [2]
Modelsim
Version 5.7a 92.9% 142 Hours 66.6%
Lesser fault coverage but reasonable test synthesis time.
Greedy GA proposed in the last chapter
GHDL 95.8% 172 Hours 66.6%
Improved fault coverage but longer test synthesis Proposed
RSBST
Modelsim
Version 10.5b 95.5% 98 Hours 80.8%
Adequate fault coverage and faster test synthesis
It may also happen that the offspring test program detects new faults but its logic simulation results may match with that of its parents. In these cases, new faults may get discarded. As most of the hard-to-detect faults are in the control components, it is quite likely that the faults in control unit and ALU control may left undetected in the RSBST scheme.
4.5 Summary