 What is MAcroExpressions Test Application
(Maestra)? ( Contents)
(Initial intro is
here. For a brief executive summary click
here).
Complete
motivation and
description (in HTML format) is available in a new window or get it
as PDF.
Maestra is a reference
implementation of a C/C++ unit test environment described in the article
"C code unit testing on a shoestring" by
Ark Khasin (and also discussed below).
It is free for use and
modification; however it comes without any warranty whatsoever.
The name stands for
MAcroExpressions Test Run Application; it is
also an Italian term for a (female) elementary school teacher. That seems quite
appropriate for a testing environment.
 Why yet
another unit test environment? (
Contents)
If your C (or C++)
software project is safety-critical (or safety-related) and you plan to certify
it according to a safety standard, you need to meet certain requirements.
Invariably, one of them is to produce unit tests documentation proving
code/branch coverage of your tests.
In embedded software, it
may become a requirement to run the unit tests on the real target (or on a
close emulator or simulator). That means that the footprint of the test must
be reasonably small. That also means using the same compiler and with
the same switches as in production code; there may also be other reasons
for it.
These are the areas where
the free and widely accepted test environments,
CUnit and
CppUnit, fall short.
They provide no proof of test coverage and include test and test result
management in the runtime environment thus increasing the footprint
dramatically. They may also require compiler adaptation (port).
There are commercial
tools that got it right:
- They use test harness
configured at compile time, so a unit test build is reasonably lean.
- They produce
plain-text output which is to be analyzed by a separate tool (definitely
running on a host machine) to extract pass/fail and code coverage
information.
- Test set management is
done by a host-based tool
- Code/branch coverage
is achieved by automatically instrumenting the code in the unit under
test, i.e., replacing the code with another, functionally equivalent but
talkative (announcing its execution trace).
So, what's wrong with
using the commercial tools?
First, they are very
expensive. Thus, they make entry threshold into safety-related product
development rather high.
Second, they are complex
software systems with complex configurations. That means a steep learning
curve.
Third, they require
adaptation to your compiler. This may not be readily available. If it is, it
may have defects.
Fourth, code
instrumentation depends on parsing the code. And if the code uses clever
constracts, there are realistic chances that it won't be parsed
correctly.
That said, there are (and
will be) people who got over those issues or learned to live with them.
However, there is a different approach possible.
 Maestra approach (
Contents)
Acknowledging that
correct code parsing is a difficult task, let's observe that you already have a
parser that can successfully parse you code whatever language extensions your
compiler provides: it's your compiler itself. If only it could be used for code
instrumentation!
Well, as it turns out,
it's possible to instrument substantial amounts of your code by means of your C
compiler alone, and without changing your code's appearance. The trick is to
abuse the C preprocessor, namely, to redefine the keywords of the
language.
There will be a couple of
of language elements that escape instrumentation. They can be dealt with in two
ways.
First, as the article
proposes, a coding standard can adopt a certain style of using the constructs,
so that instrumentation of these constructs could be achieved.
Second, a script can be
written to convert a source file to a form lending itself to instrumentation.
Note though that it is no small task: A general-purpose script should be an
almost full-blown C (or C++, as the case may be) parser. A script good only to
cover your coding policy is much easier to write, but since you are restricting
the coding policy anyway, you can restrict it to making the first approach
work.
Whichever of the two
paths you take, you get code instrumentation for free. The rest of Maestra is a
very lightweight and small-footprint test execution environment of the
instrumented code that produces text output. In that respect, it follows the
footsteps of the major commercial test tools, leaving test report analysis to
the host tools.
(Actually, it does a bit more, namely, keeping track of test
cases, test and test sets passed, accepted or failed. It can also produce HTML
- or, for that matter, XML - output to assist in evaluating manual acceptance
criteria.)
More reading is
here.
 Getting and working with Maestra (
Contents)
Maestra is mostly a
do-it-yourself method. You can download a free reference implemntation from the
download page to get started.
From time to time,
updates may be posted here. If you'd like to be notified when an update is
available, please contact info@macroexpressions.com via
email. |
