Experimental version released on February 24th, 2012.
* Made the 'test' command record the results of the executed test
cases into a SQLite database. As a side effect, 'test' now supports a
'--store' option to indicate where the database lives.
* Added the 'report' command to generate plain-text reports of the
test results stored in the database. The interface of this command is
certainly subject to change at this point.
* Added the 'db-exec' command to directly interact with the store
database.
* Issue 28: Added support for the 'require.memory' test case property
introduced in ATF 0.15.
* Renamed the user-specific configuration file from ~/.kyuarc to
~/.kyua/kyua.conf for consistency with other files stored in the
~/.kyua/ subdirectory.
* Switched to use Lutok instead of our own wrappers over the Lua C
library. Lutok is just what used to be our own utils::lua module, but
is now distributed separately.
* Removed the 'Atffile's from the source tree. Kyua is stable enough
to generate trustworthy reports, and we do not want to give the
impression that atf-run / atf-report are still supported.
* Enabled logging to stderr for our own test programs. This makes it
slightly easier to debug problems in our own code when we get a
failing test.
Experimental version released on August 24th, 2011.
The biggest change in this release is the ability for Kyua to run test
programs implemented using different frameworks. What this means is
that, now, a Kyua test suite can include not only ATF-based test
programs, but also "legacy" (aka plain) test programs that do not use
any framework. I.e. if you have tests that are simple programs that
exit with 0 on success and 1 on failure, you can plug them in into a
Kyua test suite.
Other than this, there have been several user-visible changes. The most
important are the addition of the new 'config' and 'debug' subcommands
to the 'kyua' binary. The former can be used to inspect the runtime
configuration of Kyua after parsing, and the latter is useful to
interact with failing tests cases in order to get more data about the
failure itself.
Without further ado, here comes the itemized list of changes:
* Generalized the run-time engine to support executing test programs
that implement different interfaces. Test programs that use the ATF
libraries are just a special case of this. (Issue 18.)
* Added support to the engine to run "plain" test programs: i.e. test
programs that do not use any framework and report their pass/fail
status as an exit code. This is to simplify the integration of legacy
test programs into a test suite, and also to demonstrate that the
run-time engine is generic enough to support different test
interfaces. (Issue 18.)
* Added the 'debug' subcommand. This command allows end users to tweak
the execution of a specific test case and to poke into the behavior of
its execution. At the moment, all this command allows is to view the
stdout and stderr of the command in real time (which the 'test'
command currently completely hides).
* Added the 'config' subcommand. This command allows the end user to
inspect the current configuration variables after evaluation, without
having to read through configuration files. (Issue 11.)
* Removed the test_suites_var function from configuration files. This
was used to set the value of test-suite-sepecific variables, but it
was ugly-looking. It is now possible to use the more natural syntax
'test_suites.<test-suite-name>.<variable> = <value>'. (Issue 11.)
* Added a mechanism to disable the loading of configuration files
altogether. Needed for testing purposes and for scriptability.
Available by passing the '--config=none' flag.
* Enabled detection of unused parameters and variables in the code and
fixed all warnings. (Issue 23.)
* Changed the behavior of "developer mode". Compiler warnings are now
enabled unconditionally regardless of whether we are in developer mode
or not; developer mode is now only used to perform strict warning
checks and to enable assertions. Additionally, developer mode is now
only automatically enabled when building from the repository, not for
formal releases. (Issue 22.)
* Fixed many build and portability problems to Debian sid with GCC 4.6.3
and Ubuntu 10.04.1 LTS. (Issues 20, 21, 26.)
Kyua (pronounced Q.A.) is a testing framework for both developers and
users. Kyua is different from most other testing frameworks in that it
puts the end user experience before anything else. There are multiple
reasons for users to run the tests themselves, and Kyua ensures that
they can do so in the most convenient way.
At the moment, Kyua is focused on implementing a solid foundation and a
powerful command-line tool to run tests implemented with the Automated
Testing Framework (ATF). Later on, Kyua will also provide a set of
language bindings (C, C++ and shell, at the least) to ease the
implementation of test cases in a variety of programming languages.
In effect, Kyua is intended to be a replacement for ATF.
