Status report: 2012-12-24 - 2012-12-30

Good news everyone!

• Initial kyua-testers code published. It took a long time, but it has finally happened.
• After this announcement, continued the work to clean up the new testers branch in the main kyua-cli repository. At this point, the latter has lost 5K lines of code and got another 5K lines replaced with more generic and simpler abstractions. There are more changes to come that will make things leaner, but these are starting to become tricky to implement.

Have a happy beginning of 2013 :-)

Initial kyua-testers code published

Three months, a hurricane and an apartment move after the proposal to modularize testers as independent binaries, I am pleased to announce the publication of the initial code implementing such proposal! It has been a long way full of interrupts, but this marks a major milestone in the development of Kyua.

The code for the independent testers can be found in the new kyua-testers module, which weights at ~10K lines of C code (yep, no C++) and currently provides testers for ATF-based and old-style (framework-less) test programs. As the documentation describes, the goal of these programs is to provide a scriptable and generic interface to run diverse test programs. This provides the ability to: first, plug new testers dynamically into kyua(1)(e.g. imagine a new tester for googletest-based test programs); and, second, to simplify the code of the frontend by removing tricky low-level code so that, potentially, most of kyua(1) could be latter rewritten in Lua.

The changes to make kyua-cli use kyua-testers are now in a separate branch of kyua-cli called testers, which drops ~4K lines of C++ code (much more slaying to come). While these changes already work and do not appear to introduce functional regressions, there are still two major issues to fix:

1. The first pending item is to address a serious performance regression in kyua report. This regression is understandable considering how horrendous the database querying code is and the generalization of the database schema that went in with this refactoring. Conceptually easy to fix, but the database-backing code needs some serious cleanup.
2. The second pending item is to improve the code in kyua-cli to execute the testers, which in turn will obsolete —and thus enable the removal of— lots of nasty and unnecessary abstractions. The current implementation to do this has been a hack of mine in an attempt to get things to work as quickly and as unobtrusively as possible. Now that the code is in place and that all the tests pass, it will be easier to reorganize the code.

If you inspect the code, you will realize that the engine/atf_iface/ and engine/plain_iface/ directories of kyua-cli are now gone. This is a big win because the abstractions in kyua-cli (those representing test programs, test cases and test results) are now really unaware of the specific interface implemented by the test programs being executed.

As soon as the two above items are addressed, I will start evaluating the merge of the testers branch into master, which should unblock the integration of Kyua into NetBSD! Stay tuned.

Status report: 2012-12-17 - 2012-12-23

Long time since the last report. Have been busy sorting things out post-Sandy and with a work trip... But, finally, I have been able to put some hours of work into Kyua these last few days:

• Integrated the independent testers into Kyua, and got all tests to pass. The work of the last two months if finally seeing its results!
• Started the removal of all code in the kyua binary that implemented the execution of tests in-process. There is still quite a few code that needs yanking though.
• Many cleanups to the testers code in preparation for publication: fixing pending to-dos and adding tests.

Stay tuned for the shortcoming announcement about the publication of the testers and a branch with the code to use them. Once this happens, I'll still have to address a serious performance regression in kyua report before this can hit mainline.