A Methodical View of Testing with TPT

Black-Box, Grey-Box, and White-Box Testing

Operational testing can be distinguished between these three approaches: Black-Box, Grey-Box, and White-Box testing. The differentiating factor is the information available to the tester for test creation and execution. For the embedded domain, the following characteristics apply.

Black-Box Test

In Black-Box Testing, the tester receives a test object as a black box along with a description of how the Black Box should behave, usually in the form of requirements. There is no information about the internal structure. The tester creates test cases, executes them, and compares the responses of their Black-Box with the expected values derived from the specification.

Grey-Box Test

Test case creation in Grey-Box testing is essentially like Black-Box testing, but the tester has basic knowledge of the internal structure, for example, through descriptions of internal states the system can assume. However, the tester does not have direct access to the implementation or the code.

White-Box Test

In White-Box testing, the tester has all the information, including insight into the codebase. In practice, this approach often leads to lower product quality than the Black-Box approach. From a quality assurance perspective, this approach should never be used as a reference for deriving expected values. However, there are meaningful exceptions, such as when White-Box testing is used to test defensive programming constructs, like sequentially repeating null pointer checks, which cannot be stimulated with Black-Box approaches.

Conclusion

It is unclear whether coverage reports available to the tester after execution are considered White Box or Black Box. This remains somewhat ambiguous. However, it is clear that coverage reports can help identify gaps in testing compared to requirements and/or gaps in requirements compared to the code.