+
+======================================================
+|mutest| - A simple micro unit testing framework for C
+======================================================
+
+:Author: Leandro Lucarella
+:Contact: llucax@gmail.com
+:Version: 1.0
+:Date: |date|
+:Copyright: Leandro Lucarella (2008), released under the BOLA_ license
+:Abstract: |mutest| is a micro `unit testing`_ framework for C (with some
+ `C++ support`_). It's mostly an idea (it even comes with
+ 2 implementations_ of the idea!) with the goal of being easy to use
+ (just write your `test cases`_ grouped in `test suites`_ and you're
+ set) and so small and simple that you don't mind to copy the files to
+ your project and just use it (i.e., no dependencies).
+
+ The idea is simple: a source file is a `test suite`_, a function is
+ a `test case`_ (special functions can be used for `test suite`_
+ initialization_ and termination_), which can can have several checks_.
+ Checks_ comes in 2 flavors, one that only prints an error, and one that
+ terminates the current `test case`_ too. A (normally) automated `test
+ program`_ run all the `test suites`_ and print some stats. It fails
+ (returns non-zero) if any `test suite`_ fails.
+
+
+.. contents::
+ :backlinks: entry
+
+
+Installation
+============
+
+Download the `latest distribution tarball`__ and uncompress it.
+
+__ http://proj.llucax.com.ar/home/mutest/releases/mutest.tar.gz
+
+You can also download any release from the `releases directory`__ or get it
+using Git_ directly from the `Git repository`__.
+
+__ http://proj.llucax.com.ar/home/mutest/releases/
+__ http://git.llucax.com.ar/w/software/mutest.git
+
+You can get `this manual`__ too, or a `PDF version`__ of it.
+
+__ http://proj.llucax.com.ar/home/mutest/manual.html
+__ http://proj.llucax.com.ar/home/mutest/manual.pdf
+
+To actually install |mutest| run::
+
+ $ make install
+
+Default installation path is ``/usr/local`` (because of that, you'll probably
+need superuser privileges to install to the default location). You can override
+that by passing the ``prefix`` make variable, for example::
+
+ $ make prefix=/opt/mutest install
+
+If you want to install just the docs, you can do::
+
+ $ make install-doc
+
+Or even ``install-readme``, ``install-html`` or ``install-pdf`` if you are too
+picky.
+
+If you want to install just one particular implementation_, to can use the
+``install-c`` and ``install-py`` targets.
+
+
+Quick Sample
+============
+
+You can find some samples in the sample__ directory.
+
+__ http://git.llucax.com.ar/w/software/mutest.git?a=tree;f=sample;h=d8ad4dd9c3428fef5963107c82ab6a5e34ec6e00;hb=HEAD
+
+This is an example taken from there. A simple *module* called `factorial.c`_
+with its corresponding `test suite`_ (`factorial_test.c`_).
+
+You can see some `C++ support`_ in the `exception_test.cpp`_ `test suite`_.
+
+factorial.c
+-----------
+
+.. include:: sample/factorial.c
+ :literal:
+
+factorial_test.c
+----------------
+
+.. include:: sample/factorial_test.c
+ :literal:
+
+exception_test.cpp
+------------------
+
+.. include:: sample/exception_test.cpp
+ :literal:
+
+
+Concepts
+========
+
+|mutest| is about 4 simple concepts: `test program`_, `test suite`_, `test
+case`_ and checks_. Well, to be honest you probably will need `test suite`_
+initialization_ and termination_ too =)
+
+
+Test Program
+------------
+
+A **test program** is the higher level unit of |mutest|. The test program is
+the one in charge of running all your tests. Probably one of the more important
+features of |mutest| is that you are not supposed to bother about the test
+program. So, different implementations_ have different ways to tackle this.
+Some need more or less interactions from your part, and each have their pros
+and cons.
+
+But this is all you need to know for now, for more details see how the test
+program is implemented by your implementation_ of choice.
+
+
+Test Suite
+----------
+
+A **test suite** is the higher level unit of |mutest| that you should
+care about =). Is not much more than a way to group `test cases`_. Code-wise,
+a test suite is a C (or C++) module (or compilation unit). Not clear enough?
+A unit test is an object file (could be a shared object depending on the
+implementation_). This module should have one or more `test cases`_ and it
+could have any number (including zero) of initialization_ and termination_
+functions.
+
+A test suite, is inspected by the `test program`_ for `test cases`_ and
+initialization_ and termination_ functions, and run them.
+
+A test suite fail if one or more `test cases`_ fail, and it's skipped if one or
+more initialization_ functions fail (or, depending on the implementation, if
+the test suite can't be loaded at all).
+
+
+Test Case
+---------
+
+A **test case** is just a plain function with a special signature and name.
+A test case function name must start with ``mu_test``, and take no arguments
+and return nothing. For example::
+
+ void mu_test_something(void);
+
+A test case (probably) only make sense if it has checks_. A test case succeed
+only if all its checks succeed too.
+
+Test are executed in an implementation_-dependant order, but usually the
+default order is alphabetical.
+
+
+Checks
+------
+
+Checks are assertions that a `test case`_ must pass (a boolean expression that
+must evaluate to *true*). There are 2 big flavors of checks: **check** and
+**ensure**. **check** just print an error (and *mark* the `test case`_ as
+failed) and **ensure** halt the `test case`_ execution, jumping to the next
+one.
+
+For better `C++ support`_ there are check macros that assert that a specified
+exception is thrown (instead of check for a boolean expression to evaluate to
+*true*).
+
+You can take a look at the reference_ to see the different flavors of check
+macros in more detail.
+
+
+Initialization
+--------------
+
+Sometimes you need to setup some environment shared between all the `test
+cases`_ in a `test suite`_. You can use **initialization functions** for this.
+
+An initialization function, like a `test case`_, is a plain C function with
+a special name and signature. The name must start with ``mu_init`` and it must
+take no arguments, and return an *error code* (``0`` being success). For
+example::
+
+ int mu_init_something(void);
+
+All initialization functions are executed before any `test case`_, in an
+implementation_-dependant order, and if one of them fail (returns non-zero),
+the whole `test suite`_ is skipped immediately.
+
+
+Termination
+-----------
+
+**Termination functions** are just like initialization_ functions, but they're
+executed **after** the `test cases`_, their names start with ``mu_term`` and
+they return nothing. For example::
+
+ void mu_term_something(void);
+
+
+C++ Support
+===========
+
+You can use |mutest| with C++, the only care you must take is that, because of
+C++ `name mangling`_ (and |mutest| relaying on function names), you must
+declare your `test cases`_ and initialization_ and termination_ functions as
+``extern "C"`` (see `exception_test.cpp`_ for an example).
+
+Checks_ become *exception-safe* when using |mutest| with a C++ compiler, and
+2 extra checks_ designed for C++ get defined (`mu_echeck()`_ and
+`mu_eensure()`_). They assert that an expression throws a particular exception.
+
+
+Implementations
+===============
+
+There are 2 big groups of possible implementations that I can think
+of: *static* and *dynamic*.
+
+|mutest| comes with one implementation of each group.
+
+
+Static Implementations
+----------------------
+
+Static implementations can be only written in C/C++ (or other language that is
+link-compatible with C, like the `D Programming Language`_, but since one of
+the main goals of |mutest| is avoid unnecessary dependencies, you probably
+don't want to depend on an extra language/compiler to run your tests =).
+
+The main advantage is better debugging support, because you can run the `test
+program`_ in a standard debugger and see what happens with `test cases`_ very
+naturally.
+
+The main disadvantage is, the `test suites`_ must be figured out in
+*compile-time*, usually using some kind of code generation (if you want to
+avoid writing repetitive code yourself). There's also a limitation in the `test
+case`_, initialization_ and termination_ functions names: they should be unique
+for all the `test program`_.
+
+
+C implementation
+~~~~~~~~~~~~~~~~
+
+|mutest| comes with a C static implementation. Only 3 files are needed:
+``mutest.c`` (the *user-independent* part of the `test program`_), ``mkmutest``
+(a bash script for generating the *user-dependent* part of the `test program`_)
+and ``mutest.h`` (the header file that `test suites`_ should include).
+
+You can copy this 3 files to your project or install them at system-level and
+use them globally.
+
+The procedure is simple, You should compile you `test suites`_, ``mutest.c``
+and the generated output of ``mkmutest`` as object files and link them
+together.
+
+For example::
+
+ $ cc -c -o mutest.o mutest.c
+ $ cc -c -o test1.o test1.c
+ $ cc -c -o test2.o test2.c
+ $ mkmutest mutest.h test1.o test2.o | cc -xc -c -o runmutest.o -
+ $ cc -o testprg mutest.o test1.o test2.o runmutest.o
+
+Then you can run the `test program`_ invoking it with no arguments::
+
+ $ ./testprg
+
+
+``mkmutest`` Invocation
+"""""""""""""""""""""""
+
+This small script take 1 mandatory positional argument: the path to the
+``mutest.h`` file. All remaining positional arguments should be object files
+representing `test suites`_.
+
+
+Test Program Invocation
+"""""""""""""""""""""""
+
+The test program can be invoked without arguments, but can take some extra
+options:
+
+``-v``
+ Be verbose. This is accumulative, when you add extra ``-v`` you will
+ get extra verbosity.
+
+ By default, you just get failed checks_ printed. If you use a single
+ ``-v``, a summary of failed/passed `test suites`_, `test cases`_ and
+ checks_ will be printed. If an extra ``-v`` is used, you'll see the current
+ `test suite`_ being executed. Another ``-v`` and you'll get the current
+ `test case`_, and another one, and you'll get each check_.
+
+
+Dependencies
+""""""""""""
+
+Even when dependencies are kept minimal, there always be a few ;)
+
+To use this implementation you just need:
+
+* A C compiler (you already needed that, so...)
+* The ``nm`` program (from `GNU Binutils`_, included in virtually any \*NIX)
+* The `GNU Bash`_ shell interpreter (also included in virtually any \*NIX)
+
+
+Dynamic Implementations
+-----------------------
+
+Dynamic implementations, on the other hand, can be written in any language that
+can access to shared objects. The idea is to inspect a shared object for `test
+suites`_ and run them, without requiring any information about `test suites`_
+at compile time.
+
+There are several advantages in this kind of implementations. The dynamic
+nature let you completely separate the `test program`_ from the user-written
+`test suites`_ and you can choose at *run-time* what `test suites`_ to execute
+by just selecting the correct shared objects. Also, `test case`_,
+initialization_ and termination_ functions names only have to be unique in the
+scope of the `test suites`_, because `test suites`_ are completely isolated in
+separate shared objects.
+
+But everything comes at a price, and the higher price to pay is
+*debuggability*. It's a little harder to plug a debugger to a shared object.
+
+
+Python implementation
+~~~~~~~~~~~~~~~~~~~~~
+
+This implementation is much simpler and elegant than the `C implementation`_.
+Only 2 files are needed: ``mutest`` (`test program`_ written in Python_ using
+ctypes_ module to access the shared object symbols) and ``mutest.h`` (the
+header file that `test suites`_ should include).
+
+Since both implementations provided by |mutest| share the same ``mutest.h``,
+you should define the ``MUTEST_PY`` macro when compiling the `test suites`_ if
+you will run them using this implementation.
+
+As with the `C implementation`_, you can copy this 2 files to your project or
+install them at system-level and use them globally.
+
+The procedure is even simpler than the `C implementation`_: compile and link
+you `test suites`_ as shared objects and then run the ``mutest`` program
+passing the shared objects as arguments. For example::
+
+ $ cc -c -fPIC -DMUTEST_PY -o test1.o test1.c
+ $ cc -shared -o test1.so test1.o
+ $ cc -c -fPIC -DMUTEST_PY -o test2.o test2.c
+ $ cc -shared -o test2.so test2.o
+ $ mutest test1.so test2.so
+
+That's it.
+
+``mutest`` Invocation
+"""""""""""""""""""""
+
+``mutest`` program takes `test suites`_ shared objects to run as positional
+arguments. It accepts the same options as the `C implementation's test
+program`__ and some extra options are accepted too:
+
+``--verbose``
+ Alias for ``-v``.
+
+``-q``, ``--quiet``
+ Be quiet (no output is shown at all).
+
+``-s``, ``--search``
+ Search for `test suites`_ (\*.so) in the current directory and add them
+ to the list of `test suites`_ to run.
+
+``-h``, ``--help``
+ Show a help message and exit.
+
+__ `Test Program Invocation`_
+
+
+Dependencies
+""""""""""""
+
+As with the `C implementation`_, some minor dependencies are needed:
+
+* Python_ (2.5 or later)
+* The ``nm`` program (from `GNU Binutils`_, included in virtually any \*NIX)
+
+You will need a C compiler for building the `test suites`_ too, but technically
+is not needed by |mutest| itself ;)
+
+
+Reference
+=========
+
+``mu_check()``
+--------------
+
+Synopsis
+ ``mu_check(expression)``
+
+Description
+ Check that the ``expression`` evaluates to *true*. Continue with the
+ `test case`_ if fail.
+
+Availability
+ Always
+
+Example
+ ::
+
+ void mu_test(void)
+ {
+ mu_check(5 == 4); /* fail */
+ mu_check(5 == 5); /* excecuted, pass */
+ }
+
+
+``mu_ensure()``
+---------------
+
+Synopsis
+ ``mu_ensure(expression)``
+
+Description
+ Check that the ``expression`` evaluates to *true*. Interrupt the `test
+ case`_ if fail.
+
+Availability
+ Always
+
+Example
+ ::
+
+ void mu_test(void)
+ {
+ mu_ensure(5 == 4); /* fail */
+ mu_check(5 == 5); /* not excecuted */
+ }
+
+
+``mu_echeck()``
+---------------
+
+Synopsis
+ ``mu_echeck(class, expression)``
+
+Description
+ Check that the ``expression`` throws a specific exception ``class`` (or
+ subclass). Continue with the `test case`_ if fail.
+
+Availability
+ C++ only
+
+Example
+ ::
+
+ #include <stdexcept>
+
+ extern "C"
+ {
+ void mu_test(void)
+ {
+ mu_echeck(std::exception, true); /* fail */
+ mu_echeck(std::exception,
+ throw std::runtime_error("!")); /* excecuted, pass */
+ }
+ }
+
+``mu_eensure()``
+----------------
+
+Synopsis
+ ``mu_eensure(class, expression)``
+
+Description
+ Check that the ``expression`` throws a specific exception ``class`` (or
+ subclass). Interrupt the `test case`_ if fail.
+
+Availability
+ C++ only
+
+Example
+ ::
+
+ #include <stdexcept>
+
+ extern "C"
+ {
+ void mu_test(void)
+ {
+ mu_eensure(std::exception, true); /* fail */
+ mu_echeck(std::exception,
+ throw std::runtime_error("!")); /* not excecuted */
+ }
+ }
+
+
+About
+=====
+
+This manual was written using reStructuredText_.
+
+.. Use section numbers
+.. sectnum::
+ :suffix: .
+
+.. Internal Links (aliases):
+.. _`test suites`: `test suite`_
+.. _`test cases`: `test case`_
+.. _check: checks_
+.. _implementation: implementations_
+
+
+.. External Links:
+.. _BOLA: http://blitiri.com.ar/p/bola/
+.. _`unit testing`: http://en.wikipedia.org/wiki/Unit_testing
+.. _`name mangling`: http://en.wikipedia.org/wiki/Name_mangling
+.. _`D Programming Language`: http://www.digitalmars.com/d/
+.. _`GNU Binutils`: http://www.gnu.org/software/binutils/
+.. _`GNU Bash`: http://www.gnu.org/software/bash/
+.. _Python: http://www.python.org/
+.. _ctypes: http://docs.python.org/library/ctypes.html
+.. _reStructuredText: http://docutils.sourceforge.net/rst.html
+.. _Git: http://git.or.cz/
+
+
+.. Substitutions:
+.. |mutest| replace:: *mutest*
+.. |date| date::
+
+.. vim: set filetype=rst expandtab shiftwidth=4 softtabstop=4 :
+
projects / software / mutest.git / commitdiff