Test case pattern matching

Test case pattern matching

Problem statement

A common problem in testing arises, when success of given unit test can only be formulated in terms of routine under test producing a certain output. Thus, comparison of unit test output to some predefined textual reference is provided by many test frameworks, including Boost.Test, used in this project. Unfortunately, literal comparison of test output to a fixed pattern is not very useful for a major part of difficult development scenarios.

Let's consider a fairly typical message produced by a non-trivial development helper program (such messages are hereafter referred as test space):

[236.768961] '' <0x1d34e98> loci: 0, sinks a/p: 1/1

At the very least, the values in square and angled brackets can not be matched literally to a reference pattern, because they change from one test run to another (the first value being a time stamp, the second one - some heap pointer).

Basic approach to pattern space definition

One simple approach is to augment the pattern space side of output matching with configurable parser references. Thus, a useful pattern for the test message above may look like this:

[%f] '' <%p> loci: 0, sinks a/p: 1/1

With escape character ('%'), it's literal reference ('%%') and parser reference names for some float ('f') and pointer ('p') values configurable through the matcher traits interface of appropriate sort. This approach is already way more useful than invariable literal matching. Needless to add that injected parsers must match the test space input for the overall pattern match to be considered successful.

Functionalized pattern space with constraints and memoization