summaryrefslogtreecommitdiff
path: root/libs/greatest/README.md
diff options
context:
space:
mode:
authorAldrik Ramaekers <aldrikboy@gmail.com>2025-09-14 15:27:51 +0200
committerAldrik Ramaekers <aldrikboy@gmail.com>2025-09-14 15:27:51 +0200
commit76d3082e5b89fe54acaaaaadd201ccc92f4f346e (patch)
tree32d5212d87bc0b4cab06bcbe923f7eeb019268b0 /libs/greatest/README.md
parent38019a9693375ac6719ffec43bff63774e142387 (diff)
tests for administration io
Diffstat (limited to 'libs/greatest/README.md')
-rw-r--r--libs/greatest/README.md475
1 files changed, 475 insertions, 0 deletions
diff --git a/libs/greatest/README.md b/libs/greatest/README.md
new file mode 100644
index 0000000..f1ba127
--- /dev/null
+++ b/libs/greatest/README.md
@@ -0,0 +1,475 @@
+# greatest
+
+A testing system for C, contained in 1 header file.
+
+
+## Key Features and Project Goals
+
+- **Small, Portable, Lightweight**
+
+ greatest doesn't depend on anything beyond ANSI C89, and the test
+ scaffolding should build without warnings when compiled with
+ `-Wall -Wextra -pedantic`. It is under 1,000 LOC (SLOCCount),
+ and does no dynamic allocation.
+
+- **Permissive License**
+
+ greatest is released under the [ISC License][ISC]. You can use it
+ freely, even for commercial purposes.
+
+- **Easy To Set Up**
+
+ To use, just `#include "greatest.h"` in your project. There is
+ very little boilerplate. Most features are optional.
+
+- **Un-Opinionated**
+
+ When a command-line test runner is useful, greatest can provide one,
+ but it can also run as part of other programs. It doesn't depend on
+ a particular build system or other tooling, and should accommodate a
+ variety of testing approaches. It actively avoids imposing
+ architectural choices on code under test. While greatest was
+ designed with C in mind, it attempts to be usable from C++.
+
+- **Modular**
+
+ Tests can be run individually, or grouped into suites. Suites can
+ share common setup, and can be in distinct compilation units.
+
+- **Low Friction**
+
+ Specific tests or suites can be run by name, for focused and rapid
+ iteration during development. greatest adds very little startup
+ latency.
+
+
+There are some compile-time options, and slightly nicer syntax for
+parametric testing (running tests with arguments) if compiled
+with a C99 or later language standard.
+
+I wrote a
+[blog post](http://spin.atomicobject.com/2013/07/31/greatest-c-testing-embedded/)
+with more information. While it's several years old, it's still accurate
+about the main functionality.
+
+[theft][], a related project, adds [property-based testing][pbt].
+
+[1]: http://spin.atomicobject.com/2013/07/31/greatest-c-testing-embedded/
+[theft]: https://github.com/silentbicycle/theft
+[pbt]: https://spin.atomicobject.com/2014/09/17/property-based-testing-c/
+[ISC]: https://opensource.org/licenses/ISC
+
+## Basic Usage
+
+```c
+#include "greatest.h"
+
+/* A test runs various assertions, then calls PASS(), FAIL(), or SKIP(). */
+TEST x_should_equal_1(void) {
+ int x = 1;
+ /* Compare, with an automatic "1 != x" failure message */
+ ASSERT_EQ(1, x);
+
+ /* Compare, with a custom failure message */
+ ASSERT_EQm("Yikes, x doesn't equal 1", 1, x);
+
+ /* Compare, and if they differ, print both values,
+ * formatted like `printf("Expected: %d\nGot: %d\n", 1, x);` */
+ ASSERT_EQ_FMT(1, x, "%d");
+ PASS();
+}
+
+/* Suites can group multiple tests with common setup. */
+SUITE(the_suite) {
+ RUN_TEST(x_should_equal_1);
+}
+
+/* Add definitions that need to be in the test runner's main file. */
+GREATEST_MAIN_DEFS();
+
+int main(int argc, char **argv) {
+ GREATEST_MAIN_BEGIN(); /* command-line options, initialization. */
+
+ /* Individual tests can be run directly in main, outside of suites. */
+ /* RUN_TEST(x_should_equal_1); */
+
+ /* Tests can also be gathered into test suites. */
+ RUN_SUITE(the_suite);
+
+ GREATEST_MAIN_END(); /* display results */
+}
+```
+
+Output:
+
+```sh
+$ make simple && ./simple
+cc -g -Wall -Werror -pedantic simple.c -o simple
+
+* Suite the_suite:
+.
+1 test - 1 passed, 0 failed, 0 skipped (5 ticks, 0.000 sec)
+
+Total: 1 test (47 ticks, 0.000 sec), 3 assertions
+Pass: 1, fail: 0, skip: 0.
+```
+
+Tests are run with `RUN_TEST(test_name)`, which can be called directly
+from the test runner's `main` function or grouped into suites (which are
+run with `RUN_SUITE(suite_name)`). (Calls to `RUN_TEST` from inside
+another test are ignored.)
+
+Test cases can be run with arguments: `RUN_TEST1(test_name, arg)` passes
+a single argument, and if C99 features are supported, then
+`RUN_TESTp(test_name, ...)` uses `__VA_ARGS__` to run a test case with
+one or mare arguments. `greatest_set_test_suffix` sets a name suffix, so
+output from the test runner can include info about arguments.
+
+Test cases should call assertions and then end with `PASS()`, `SKIP()`,
+`FAIL()`, or their custom message variants (e.g. `SKIPm("TODO");`).
+If there are any test failures, the test runner will return 1,
+otherwise it will return 0. (Skips do not cause the test runner to
+report failure.)
+
+`PASS()`, `SKIP()`, `FAIL()`, and their custom message variants are
+macros that updating internal bookkeeping and then returning and enum
+value, such as `GREATEST_TEST_RES_FAIL`. They all `return` from the
+current test case function.
+
+`PASS()`/`PASSm("msg")` prints as a dot when verbosity is zero, or
+the test name and custom message (if any) with verbosity >= 1.
+
+`FAIL()`/`FAILm("msg")` always prints "FAIL test_name: msg file:line".
+
+`SKIP()`/`SKIPm("msg")` prints as an 's' when verbosity is zero, or the
+test name and custom message (if any) with verbosity >= 1. Because skips
+are not treated as a failure by the test runner, they can be used to
+skip test cases that aren't relevant in a particular build or
+environment, a way to temporarily disable broken tests, or as a sort of
+todo list for tests and functionality under active development.
+
+Tests and suites are just functions, so normal C scoping rules apply.
+For example, a test or suite named `main` will have a name collision.
+
+(For more examples, look at `example.c` and `example_suite.c`.)
+
+
+## Filtering By Name
+
+greatest runs all tests by default, but can be configured to only run
+suites or tests whose names contain a filter string, and/or exclude
+tests whose name contains a filter string. When test name filtering and
+exclusion are used together, exclusion takes precedence.
+
+ void greatest_set_suite_filter(const char *name);
+ void greatest_set_test_filter(const char *name);
+ void greatest_set_test_exclude(const char *name);
+
+These correspond to the following command line test runner options:
+
+ `-s SUITE`: Only run suites whose names contain the string "SUITE"
+ `-t TEST`: Only run tests whose names contain the string "TEST"
+ `-x EXCLUDE`: Exclude tests whose names contain the string "EXCLUDE"
+
+For example, to run any tests with "tree" in the name, in suites with
+"pars" in the name (such as "parser"), but exclude any tests whose names
+also contain "slow":
+
+ ./test_project -s pars -t tree -x slow
+
+The string matching includes optional test name suffixes.
+
+The `greatest_set_exact_name_match()` function and corresponding `-e`
+command line runner flag can be used to only run tests and/or suites
+whose names exactly match the name filter(s). Note: exact-match suite
+filtering by name will not skip tests that are run outside of any suite.
+
+
+## Available Assertions
+
+Assertions fail the current test unless some condition holds. All
+assertions have a custom message variant (with an `m` suffix), which
+takes a message string as its first argument. For example, the
+assertion `ASSERT_EQ(apple, orange);` could instead be used like
+`ASSERT_EQm("these should match", apple, orange)`. Non-message
+assertions create a default message.
+
+
+### `ASSERT(COND)`
+
+Assert that `COND` evaluates to a true (non-zero) value.
+
+
+### `ASSERT_FALSE(COND)`
+
+Assert that `COND` evaluates to a false (zero) value.
+
+
+### `ASSERT_EQ(EXPECTED, ACTUAL)`
+
+Assert that `EXPECTED == ACTUAL`. To print the values if they
+differ, use `ASSERT_EQ_FMT`. To compare with custom equality test
+and print functions, use `ASSERT_EQUAL_T` instead.
+
+
+### `ASSERT_NEQ(EXPECTED, ACTUAL)`
+
+Assert that `EXPECTED != ACTUAL`.
+
+
+### `ASSERT_GT(EXPECTED, ACTUAL)`
+
+Assert that `EXPECTED > ACTUAL`.
+
+
+### `ASSERT_GTE(EXPECTED, ACTUAL)`
+
+Assert that `EXPECTED >= ACTUAL`.
+
+
+### `ASSERT_LT(EXPECTED, ACTUAL)`
+
+Assert that `EXPECTED < ACTUAL`.
+
+
+### `ASSERT_LTE(EXPECTED, ACTUAL)`
+
+Assert that `EXPECTED <= ACTUAL`.
+
+
+### `ASSERT_EQ_FMT(EXPECTED, ACTUAL, FORMAT)`
+
+Assert that `EXPECTED == ACTUAL`. If they are not equal, print their
+values using `FORMAT` as the `printf` format string.
+
+For example: `ASSERT_EQ_FMT(123, result, "%d");` will call `printf`
+like `printf("Expected: %d\nGot: %d\n", 123, result);` if its
+`EXPECTED` and `ACTUAL` arguments don't match.
+
+Note: `EXPECTED` and `ACTUAL` will be evaluated more than once on
+failure, so they should not be a function call with side effects.
+(Since their type is not known by the macro, they cannot be
+captured in a local variable. `typeof` is a GCC extension.)
+
+
+### `ASSERT_IN_RANGE(EXPECTED, ACTUAL, TOLERANCE)`
+
+Assert that `ACTUAL` is within `EXPECTED` +/- `TOLERANCE`, once the
+values have been converted to a configurable floating point type
+(`GREATEST_FLOAT`).
+
+greatest does not depent on floating point math.
+It is only used within `ASSERT_IN_RANGE`'s macro expansion.
+
+
+### `ASSERT_STR_EQ(EXPECTED, ACTUAL)`
+
+Assert that the strings are equal
+(i.e., `strcmp(EXPECTED, ACTUAL) == 0`).
+
+
+### `ASSERT_STRN_EQ(EXPECTED, ACTUAL, SIZE)`
+
+Assert that the first `SIZE` bytes of the strings are equal
+(i.e., `strncmp(EXPECTED, ACTUAL, SIZE) == 0`).
+
+
+### `ASSERT_MEM_EQ(EXPECTED, ACTUAL, SIZE)`
+
+Assert that the first `SIZE` bytes of memory pointed to by `EXPECTED`
+and `ACTUAL` are equal. If their memory differs, print a hexdump and
+highlight the lines and individual bytes which do not match.
+
+
+### `ASSERT_ENUM_EQ(EXPECTED, ACTUAL, ENUM_STR_FUN)`
+
+Assert that the enum value `EXPECTED` is equal to `ACTUAL`. If not,
+convert each enum value to a string using `ENUM_STR_FUN` before printing
+them.
+
+`ENUM_STR_FUN` should have a type like:
+
+ const char *FUN(int x);
+
+
+### `ASSERT_EQUAL_T(EXPECTED, ACTUAL, TYPE_INFO, UDATA)`
+
+Assert that `EXPECTED` and `ACTUAL` are equal, using the
+`greatest_equal_cb` function pointed to by `TYPE_INFO->equal` to compare
+them. The assertion's `UDATA` argument can be used to pass in arbitrary
+user data (or `NULL`) to the callbacks. If the values are not equal and
+the `TYPE_INFO->print` function is defined, it will be used to print an
+"Expected: X, Got: Y" message.
+
+
+### `ASSERT_OR_LONGJMP(COND)`
+
+Assert that `COND` evaluates to a true value. If not, then use
+`longjmp(3)` to immediately return from the test case and any
+intermediate function calls.
+
+If built with `GREATEST_USE_LONGJMP` `#define`d to 0, then all
+setjmp/longjmp-related functionality will be compiled out. This also
+reduces memory usage by `sizeof jmp_buf`, which may be several hundred
+bytes, depending on the platform.
+
+
+## Random Shuffling
+
+Groups of suites or tests can be run in shuffled order by using
+`GREATEST_SHUFFLE_SUITES` and `GREATEST_SHUFFLE_TESTS`, respectively.
+This can help find and eliminate accidental coupling between tests.
+
+The shuffling depends on the seed and the test/suite count, so a
+consistent seed will only lead to reproducible ordering until the
+group's count changes.
+
+Shuffling suites:
+
+ SHUFFLE_SUITES(seed, {
+ RUN_SUITE(suite1);
+ RUN_SUITE(suite2);
+ RUN_SUITE(suite3);
+ RUN_SUITE(suite4);
+ RUN_SUITE(suite5);
+ });
+
+Shuffling tests:
+
+ SHUFFLE_TESTS(seed, {
+ RUN_TEST(test_a);
+ RUN_TEST1(test_b, 12345);
+ RUN_TEST(test_c);
+ RUN_TESTp(test_d, "some_argument");
+ RUN_TEST(test_e);
+ });
+
+Note: Any other code inside the block will be executed several times.
+The shuffling macro expands to a loop with (count + 1) iterations -- the
+first pass counts, and the following passes only execute the next chosen
+suite/test. In particular, avoid running tests directly inside of a
+`SHUFFLE_SUITES` block without a suite, because the test will be run
+on each iteration.
+
+
+## Test Name Suffixes
+
+`greatest_set_test_suffix` can be used to set an optional name suffix
+for the next test:
+
+ for (i = 0; i < row_count; i++) {
+ const struct table_row *row = &table[row_count];
+ greatest_set_test_suffix(row->name);
+ RUN_TEST1(test_with_arg, row);
+ }
+
+This will cause the test name to print with a `_` separator and the
+suffix in all pass/fail/skip messages (i.e., `test_with_arg_KEY`). This
+is especially useful when running a test several times with different
+arguments, in shuffled order. The name suffix is included when using
+name-based filtering.
+
+The test name and optional suffix are copied into an internal buffer.
+Its size can be configured by `#define`ing the constant
+`GREATEST_TESTNAME_BUF_SIZE`. (If not `#define`d, it defaults to 128
+bytes.) If the buffer is not large enough for the name and suffix, it
+will truncate after `size - 1` bytes, to ensure that it is properly
+`\0`-terminated.
+
+The name suffix pointer is cleared after each `RUN_TEST*` call, so a
+suffix can be constructed in a stack allocated buffer without later
+dereferencing a pointer that has gone out of scope.
+
+
+## Sub-Functions
+
+Because of how `PASS()`, `ASSERT()`, `FAIL()`, etc. are implemented
+(returning a test result enum value), calls to functions that use them
+directly from test functions must be wrapped in `CHECK_CALL`:
+
+ TEST example_using_subfunctions(void) {
+ CHECK_CALL(less_than_three(5));
+ PASS();
+ }
+
+This is only necessary if the called function can cause test failures.
+The function should have a return type of `enum greatest_test_res`.
+
+
+## Command Line Options
+
+Test runners build with the following command line options:
+
+ Usage: (test_runner) [-hlfave] [-s SUITE] [-t TEST] [-x EXCLUDE]
+ -h, --help print this Help
+ -l List suites and tests, then exit (dry run)
+ -f Stop runner after first failure
+ -a Abort on first failure (implies -f)
+ -v Verbose output
+ -s SUITE only run suite w/ name containing substring SUITE
+ -t TEST only run test w/ name containing substring TEST
+ -e only run exact name match for -s or -t
+ -x EXCLUDE exclude tests containing string substring EXCLUDE
+
+Any arguments after `--` will be ignored.
+
+If you want to run multiple test suites in parallel, look at
+[parade](https://github.com/silentbicycle/parade).
+
+These command line options are processed by `GREATEST_MAIN_BEGIN();`.
+
+
+## Running Tests In Another Program
+
+Rather than producing a command line test runner (which checks the
+command line options, and exits with a pass/fail return code after
+running tests), greatest can be used more like a library. Instead of
+using `GREATEST_MAIN_BEGIN()`, use `GREATEST_INIT()` to (re-)initialize
+greatest, then use either `GREATEST_PRINT_REPORT()` to print the report
+to `GREATEST_STDOUT`, or use `greatest_get_report(&report)` to get the
+pass, fail, skip, and assertion counters.
+
+The command line flags above have corresponding functions:
+
+- `greatest_stop_at_first_fail()`
+- `greatest_abort_on_fail()`
+- `greatest_list_only()`
+- `greatest_set_exact_name_match()`
+- `greatest_set_suite_filter(const char *filter)`
+- `greatest_set_test_filter(const char *filter)`
+- `greatest_set_test_exclude(const char *filter)`
+- `greatest_get_verbosity()`
+- `greatest_set_verbosity(unsigned int verbosity)`
+
+
+## Aliases
+
+Most of the macros have prefixed and unprefixed forms. For example,
+`SUITE` is the same as `GREATEST_SUITE`.
+
+Check the source for the list -- search for `GREATEST_USE_ABBREVS`.
+
+These aliases can be disabled by `#define`-ing `GREATEST_USE_ABBREVS` to 0.
+
+
+## Color Output
+
+If you want color output (`PASS` in green, `FAIL` in red, etc.), you can
+pipe the output through the included `greenest` script in `contrib/`:
+
+```sh
+$ ./example -v | greenest
+```
+
+(Note that `greenest` depends on a Unix-like environment.)
+
+greatest itself doesn't have built-in coloring to stay small and portable.
+
+
+## TAP Format
+
+There is an awk script provided, `contrib/entapment`, that converts the
+verbose output from the default CLI test runner to TAP version 13
+format:
+
+ ./example -v | contrib/entapment