The lua-based test harness is pretty nice, but it could be nicer. Here are some ways we could improve it:

  • /CleanUpTestNames
  • When you use check(EXPRESSION) and it fails, what you get in the logfile is unhelpful
  • The command form of check has too many arguments, and is much too clever about type overloading
  • indir() doesn't work the way one might expect it to
  • The default test workspace should be a subdirectory of the test dir, not the test dir (there's all sorts of clutter in the test dir)
  • Better tools for pattern matching command output
  • Run more than one test in parallel on multi-CPU machines (extra points for noticing being run under make -j and participating in the gmake "job server" algorithm)
  • Nathaniel wants a "run the most recently modified test" option
  • Some way of re-running all failed tests would be good

See also OldTestHarnessIssues from the days of autotest.

Quick Links:    -     Downloads    -     Documentation    -     Wiki    -     Code Forge    -     Build Status