Automating Tests in C++

posted on Jan 26, 2011 at 8:29pm with no comments and tagged with C++

Recently I have been doing a lot of C++ programming. Some of the projects I was and currently am working on include:

Through these projects, I've exposed myself to more C++ and I am finding that I (gasp!) like using it in some of my programming activities. That is not to say that C++ is my tool of choice for all projects, but for the above three it certainly was.

Most recently, I have been working on the FLTL and part of the requirements of the FLTL is that it work on a number of platforms. There are usually two main problems that come up when I try to make this work:

  • Compiler errors.
  • Unexpected runtime errors.
In this post I will not address the former as it is less interesting. Instead, I will talk about a very simple testing system I made to help me discover the latter. Usually I do not write many tests that I can have automatically run. This is likely a failing on my part; sometimes I don't write tests because I'm lazy and other times because that would mean I'm not working directly on the code that I really want to be working on. In any event, it was necessary to write tests for this project as I discovered some some bugs that manifest on the various platforms on which I am testing my code.

To start things off, the following code illustrates how I use the system.

fltl/test/cfg/CFG.hpp 
#ifndef FLTL_CFG_HPP_
#define FLTL_CFG_HPP_

#include "fltl/test/Test.hpp"
#include "fltl/lib/CFG.hpp"

namespace fltl { namespace test { namespace cfg {

    FLTL_TEST_CATEGORY(test_equivalence_relations,
        "Test for equivalence of variables, terminals, symbols, and symbol strings."
    );

    FLTL_TEST_CATEGORY(test_productions,
        "Test that productions are correctly added to the grammar and that duplicates are ignored."
    );
}}}

#endif /* FLTL_CFG_HPP_ */
fltl/test/cfg/CFG.cpp 
#include "fltl/test/cfg/CFG.hpp"

namespace fltl { namespace test { namespace cfg {

    void test_equivalence_relations(void) throw() {

        using fltl::lib::CFG;

        CFG<char> cfg;

        CFG<char>::var_t S(cfg.add_variable());
        CFG<char>::var_t T(cfg.add_variable());

        CFG<char>::term_t a(cfg.get_terminal('a'));

        /* ... */

        CFG<char>::sym_t S_sym(S);
        CFG<char>::sym_t a_sym(a);

        FLTL_TEST_EQUAL(S,S);
        FLTL_TEST_EQUAL(S, S_sym);
        FLTL_TEST_EQUAL(S_sym, S);
        FLTL_TEST_NOT_EQUAL(S, T);
        FLTL_TEST_NOT_EQUAL(S, a);
        FLTL_TEST_NOT_EQUAL(a, S);

        /* ... */
    }
}}}
main.cpp 
#include "fltl/test/Test.hpp"
#include "fltl/test/cfg/CFG.hpp"

int main(void) {

    fltl::test::run_tests();

    return 0;
}

Hosted by imgur.com

As is shown above, test cases are contained within functions. A function can contain as many tests cases as one likes. The idea is that the function represents a category of test cases. Then, to add a category of test cases to the system, one simply declares (in a header file) that a function is a test category (using the FLTL_TEST_CATEGORY macro).

The way this works is fairly straightforward:

  • Test cases are basically assertions. If the condition of the test case is true then the test succeeds, otherwise the test fails. Successes and failures are recorded and printed out.
  • Each test category declaration declares the function and also a static variable whose type is parameterized over the function. When static variable constructors are called at runtime, the test case category types chain themselves into a linked list.
  • When we want to run the tests, we invoke fltl::test::run_tests() and that loops through the linked list and invokes each test case category function.

The code that includes the testing macros and types can be found here:


Comments

There are no comments, but you look like you have something on your mind.


Comment