Release date: May 1st, 2008
Status: Experimental
* Clauses 3 and 4 of the BSD license used by the project were dropped.
All the code is now under a 2-clause BSD license compatible with the
GNU General Public License (GPL).
* Added a C-only binding so that binary test programs do not need to be
tied to C++ at all. This binding is now known as the atf-c library.
* Renamed the C++ binding to atf-c++ for consistency with the new atf-c.
* Renamed the POSIX shell binding to atf-sh for consistency with the new
atf-c and atf-c++.
* Added a -w flag to test programs through which it is possible to specify
the work directory to be used. This was possible in prior releases by
defining the workdir configuration variable (-v workdir=...), but was a
conceptually incorrect mechanism.
* Test programs now preserve the execution order of test cases when they
are given in the command line. Even those mentioned more than once are
executed multiple times to comply with the user's requests.
Changes:
* Added two new manual pages, atf-c++-api and atf-sh-api, describing the
C++ and POSIX shell interfaces used to write test programs.
* Added a pkg-config file, useful to get the flags to build against the
C++ library or to easily detect the presence of ATF.
* Added a way for test cases to require a specific architecture and/or
machine type through the new 'require.arch' and 'require.machine'
meta-data properties, respectively.
* Added the 'timeout' property to test cases, useful to set an upper-bound
limit for the test's run time and thus prevent global test program stalls
due to the test case's misbehavior.
* Added the atf-exec(1) internal utility, used to execute a command after
changing the process group it belongs to.
* Added the atf-killpg(1) internal utility, used to kill process groups.
* Multiple portability fixes. Of special interest, full support for SunOS
(Solaris Express Developer Edition 2007/09) using the Sun Studio 12 C++
compiler.
* Fixed a serious bug that prevented atf-run(1) from working at all under
Fedora 8 x86_64. Due to the nature of the bug, other platforms were
likely affected too.
* Added XML output support to atf-report. This is accompanied by a DTD for
the format's structure and sample XSLT/CSS files to post-process this
output and convert it to a plain HTML report.
* Changed atf-run to add system information to the report it generates.
This is currently used by atf-report's XML output only, and is later
printed in the HTML reports in a nice and useful summary table. The user
and system administrator are allowed to tune this feature by means of
hooks.
* Removed the test cases' 'isolated' property. This was intended to avoid
touching the file system at all when running the related test case, but
this has not been true for a long while: some control files are
unconditionally required for several purposes, and we cannot easily get
rid of them. This way we remove several critical and delicate pieces of
code.
* Improved atf-report's CSV output format to include information about
test programs too.
* Fixed the tests that used atf-compile to not require this tool as a
helper. Avoids systems without build-time utilities to skip many tests
that could otherwise be run. (E.g. NetBSD without the comp.tgz set
installed.)
* Many general cleanups: Fixed many pieces of code marked as ugly and/or
incomplete.
* Test cases now get a known umask on entry.
* atf-run now detects many unexpected failures caused by test programs
and reports them as bogus tests. atf-report is able to handle these
new errors and nicely reports them to the user.
* All the data formats read and written by the tools have been
documented and cleaned up. These include those grammars that define
how the different components communicate with each other as well as
the format of files written by the developers and users: the Atffiles
and the configuration files.
* Added the atf-version tool, a utility that displays information about
the currently installed version of ATF.
* Test cases can now define an optional cleanup routine to undo their
actions regardless of their exit status.
* atf-report now summarizes the list of failed (bogus) test programs
when using the ticker output format.
* Test programs now capture some termination signals and clean up any
temporary files before exiting the program.
* Multiple bug fixes and improvements all around.
The Automated Testing Framework (ATF) is a collection of libraries and
utilities designed to ease unattended application testing in the hands of
developers and end users of a specific piece of software.
As regards developers, ATF provides the necessary means to easily create
test suites composed of multiple test programs, which in turn are a
collection of test cases. It also attempts to simplify the debugging of
problems when these test cases detect an error by providing as much
information as possible about the failure.
As regards users, it simplifies the process of running the test suites and,
in special, encourages end users to run them often: they do not need to
have source trees around nor any other development tools installed to be
able to certify that a given piece of software works on their machine as
advertised.
Yes, these are (part of) the results of my SoC 2007 project :-)
