Synopsys Cloud
Cloud native EDA tools & pre-optimized hardware platforms
Request a Free Trial →
Table of Contents
While MC/DC may be a common term in software testing, its correct use and safety-relevant role still calls for clarification throughout the industry.
Since testing 100% of all variants is simply not possible, testing is always a compromise. Limited time and budget don’t allow testing a software completely. And yet, it is testing that creates confidence in the quality of software. The challenges in testing are the allocation of resources and how they are used optimally. The solution lies in planning and prioritization of your test objectives with a well-chosen “definition of done” and your test organization is more likely to be successful.
Planning is based on test objectives. These can have varying degrees of granularity. Start with a general definition of a test organization, and give details per test object on each test level, and per release. Essentially, the definition of test objectives involves balancing. That is, deciding what should and should not be tested. The development status of the product and the boundary conditions have the biggest influence in determining the objectives.
Meanwhile, safety standards have to be complied with. Standards play an important role in software testing, particularly in the case of safety-relevant products. They set high requirements for the verification of safety-relevant software. ISO26262-6 states that requirements coverage and structure coverage must be collected by suitable coverage metrics. This serves as an evaluation of the completeness of the verification. For software with the highest safety rating (ASIL D), modified condition & decision coverage (MC/DC) at unit level is highly recommended.
One might think that this makes MC/DC a test objective. But this is wrong. Test objectives should by definition prove properties of the software under test. The correct functionality of a unit should always be at the top of the list of test objectives. MC/DC shows whether all possible decisions and conditions have been passed. However, it cannot by itself prove whether the system behaves correctly. Therefore, coverage is not suitable as a test objective.
In fact, coverage metrics should only be used as test end criteria. End-of-test criteria define when a system under test is considered to be sufficiently fully tested. Test objectives and test end criteria are defined in test concepts. It is recommended to update the test concept per iteration/ release with regard to the implemented changes and their possible effects.
A Successful Approach to Testing Needs Structure
Start by defining test cases based on requirements. Represent requirements as use cases and use methods, such as boundary value consideration or building of equivalence classes. This will help verify an ideally complete functionality of your software. At any rate, this is a solid starting point for your test. By measuring code coverage, you can discover possible vulnerabilities that have not yet been tested, and write the corresponding test cases.
The target value of the coverage must always be 100%. ISO 26262 requires an explanation for deviating values. If you have unreachable program parts, for example for debugging or parallel software configurations, we recommend that you explain the reduced coverage in the report instead of setting the target value for coverage lower in advance. You will be more efficient on the whole, because you don’t have to check and adjust your chosen reduced coverage value by complicated calculations every time you change the unit.
If you have problems reaching 100% coverage with the above-mentioned procedure, this can be due to several reasons:
1. Requirements are missing or incomplete
2. Test cases are not sufficient
3. Test cases identified dead, unreachable, or disabled code, or unintended functionalities
Since ISO 26262 requires a justification for each deviation, visualizing the relevant sections of code (see Figure 1) can be a quick way to separate the intended causes from the problems.
Figure 1: Example of MC/DC visualization in code – for variable ‘C’ the MC/DC is not complete
Testing is heavily dependent on the quality of the requirements and the chosen architecture and design of the software. In order to keep testing efforts as low as possible, it is advisable to already know the effects on the testing process in the software architecture and software design. Chose suitable architecture and design patterns.
Communication is the key to success here – software architects and designers have the task of looking at the entire software product over the lifecycle and have the opportunity to exert significant influence on hedging activities through restructuring and separation. If they are not already in contact, they should seize the opportunity and enter into exchange and discussion.
Continue Reading
Blog
2 min read
/
Sep 12, 2024
Industry First Verification IP for Arm AMBA DTI-G
