lest – lest errors escape testing

This tiny C++11 test framework is based on ideas and examples by Kevlin Henney [1,2] and on ideas found in the CATCH test framework by Phil Nash [3].

Let writing tests become irresistibly easy and attractive.

Contents

• Example usage
• In a nutshell
• License
• Dependencies
• Installation
• Usage
• Synopsis
• Variants of lest
• Features of lest
• Reported to work with
• Building tests and examples
• Contributions to lest
• Other test frameworks
• Notes and references

Example usage

#include "lest/lest.hpp"

using namespace std;

const lest::test specification[] =
{
    CASE( "Empty string has length zero (succeed)" )
    {
        EXPECT( 0 == string(  ).length() );
        EXPECT( 0 == string("").length() );
    },

    CASE( "Text compares lexically (fail)" )
    {
        EXPECT( string("hello") > string("world") );
    },

    CASE( "Unexpected exception is reported" )
    {
        EXPECT( (throw std::runtime_error("surprise!"), true) );
    },

    CASE( "Unspecified expected exception is captured" )
    {
        EXPECT_THROWS( throw std::runtime_error("surprise!") );
    },

    CASE( "Specified expected exception is captured" )
    {
        EXPECT_THROWS_AS( throw std::bad_alloc(), std::bad_alloc );
    },

    CASE( "Expected exception is reported missing" )
    {
        EXPECT_THROWS( true );
    },

    CASE( "Specific expected exception is reported missing" )
    {
        EXPECT_THROWS_AS( true, std::runtime_error );
    },
};

int main( int argc, char * argv[] )
{
    return lest::run( specification, argc, argv );
}

Note: besides above table approach, lest also supports auto-registration of tests.

Compile and run

prompt>g++ -Wall -Wextra -std=c++11 -I../include -o 05_select.exe 05_select.cpp && 05_select.exe
05_select.cpp:17: failed: Text compares lexically (fail): string("hello") > string("world") for "hello" > "world"
05_select.cpp:22: failed: got unexpected exception with message "surprise!": Unexpected exception is reported: (throw std::runtime_error("surprise!"), true)
05_select.cpp:37: failed: didn't get exception: Expected exception is reported missing: true
05_select.cpp:42: failed: didn't get exception of type std::runtime_error: Specific expected exception is reported missing: true
4 out of 7 selected tests failed.

With Buck:

prompt> buck run example/:05_select
...

In a nutshell

lest is a small C++11 test framework for unit testing, regression testing, Test-driven development (TDD) and Behaviour-driven design (BDD). It replicates innovative ideas in C++ testing from the Catch test framework such as function-level fixtures and expression-decomposing assertion macros in a form that is compact enough to read in five minutes. The lest_cpp03 variant provides similar capabilities to use with C++98/03 compilers.

Features and properties of lest are ease of installation (single header), no boilerplate code, traditional unit test cases and BDD style scenarios, strings as test names, function-level fixtures, expression-decomposing assertion macros, support for floating point comparison, test selection from commandline, test duration timing, test randomisation and sorting, display of passing tests, colourised output (compile-time option), C++11 code and a C++98/03 variant with comparable features (also compilable as C++11).

Features available via other projects are mocking (see integrate Trompeloeil mocking framework) and hamcrest matchers (see variants of lest),

Not provided are things present in other test frameworks, such as suites of tests, value-parameterised tests, type-parameterised tests, test data generators, customisable reporting, easy logging of extra information, breaking into a debugger, concurrent execution of tests, isolated execution of tests, Visual Studio Test Adapter.

License

lest uses the Boost Software License.

Dependencies

lest has no other dependencies than the C++ standard library.

Installation

lest is a single-file header-only library. Put lest.hpp, or a variant of it such as lest_cpp03.hpp directly into the project source tree or somewhere reachable from your project.

Usage

• Making a case – example
• Asserting for success – example
• Asserting for failure – example
• Asserting for exceptions – example
• Asserting for a specific exception – example
• Comparing floating point numbers – example.
• Auto test registration – single-file example
• Auto test registration with multiple source files – example part 1, 2, 3.
• Non-auto test registration – single-file example
• Non-auto test registration with multiple source files – example part 1, 2, 3.
• Using fixtures – example.
• Using lest assertions in a (reusable) user-defined function or lambda – function, , .

Synopsis

Contents

• Command line
• Test case macro
• Fixture macros
• Assertion macros
• BDD style macros
• Module registration macro
• Feature selection macros
• Standard selection macro
• Namespace
• Tests
• Main
• Main (Trompeloeil)
• Floating point comparison
• Reporting a user-defined type

Command line

Usage: test [options] [test-spec ...]

Options:

• -h, --help, this help message
• -a, --abort, abort at first failure
• -c, --count, count selected tests
• -g, --list-tags, list tags of selected tests
• -l, --list-tests, list selected tests
• -p, --pass, also report passing tests
• -z, --pass-zen, ... without expansion
• -t, --time, list duration of selected tests
• -v, --verbose, also report passing or failing sections
• --order=declared, use source code test order (default)
• --order=lexical, use lexical sort test order
• --order=random, use random test order
• --random-seed=n, use n for random generator seed
• --random-seed=time, use time for random generator seed
• --repeat=n, repeat selected tests n times (-1: indefinite)
• --version, report lest version and compiler used
• --, end options

Test specification:

• "@", "*": all tests, unless excluded
• empty: all tests, unless tagged [hide] or [.optional-name]
• "text": select tests that contain text (case insensitive)
• "!text": omit tests that contain text (case insensitive)

Test descriptions can contain tags such as [option], [hide] and [.integration]. Tests that contain the tag [hide] or a tag that starts with [. in their description are skipped, unless they are specifically selected by specifying "@", "*", or by specifying (part of) the tag.

Test specifications can be combined and are evaluated left-to-right. For example: a !ab abc selects all tests that contain 'a', except those that contain 'ab', but include those that contain 'abc'.

When regular expression selection has been enabled (and works), test specifications can use the regular expression syntax of std::regex_search(). See also lest_FEATURE_REGEX_SEARCH in section Other Macros.

Test case macro

A lest test specification can consist of a) one or more arrays of test cases that use lambdas, or b) auto-registered test cases that use free functions. See also macro lest_FEATURE_AUTO_REGISTER.

CASE( "proposition" ) { code }   (array of cases)
Describe the expected behaviour to test for and specify the actions and expectations. See also section Module registration macro – Single-file code example – Multi-file code example part 1, 2, 3.

CASE_ON( "proposition", ...) { code }   (array of cases, since v1.33)
Describe the expected behaviour to test for and specify the actions and expectations. After the description you can add a lambda capture list to refer to symbols in the enclosing scope. See also section Module registration macro – Single-file code example – Multi-file code example part 1, 2, 3.

lest_CASE( specification, "_propos