freebsd/tools/regression/TODO

$FreeBSD$
---

A lot of the tests written in C use assert(), only printing 'ok' at the end
of the program if none of the assert()s fired.

This is wrong.  They should do the test outside of an assert(), and print
'ok' or 'not ok' as appropriate, instead of bailing out part way through.
That way if a test starts failing we'll know exactly which one failed, and
the other tests can continue.

This probably means a simple libtest.so that they can link against to
provide an ok() function (the non-fatal assert), probably with some helper
functions to keep track of test numbers and so on.

---

Any of the tests written in C should link against the libraries under
/usr/obj rather than using the system libraries.

---

Not everything's been converted to the new style.  In particular;

   atm/
   ccd/
   some of geom_*/
   some of netinet/
   nfsmmap/
   p1003_1b/
   some of pipe/
   security/
   some of sockets/
   tls/

needs to be converted.  And fsx/ and gaithrstress/ are (I think) standalone
programs rather than individual tests, so aren't amenable to this sort of 
conversion.
Switch over to a different, more flexible test output protocol that's understood by Perl's Test::Harness module and prove(1) commands. Update README to describe the new protocol. The work's broken down into two main sets of changes. First, update the existing test programs (shell scripts and C programs) to produce output in the ok/not ok format, and to, where possible, also produce a header describing the number of tests that are expected to be run. Second, provide the .t files that actually run the tests. In some cases these are copies of, or very similar too, scripts that already existed. I've kept the old scripts around so that it's possible to verify that behaviour under this new system (in terms of whether or not a test fails) is identical to the behaviour under the old system. Add a TODO file. 2004-11-11 19:47:55 +00:00			$FreeBSD$
			`---`

			`A lot of the tests written in C use assert(), only printing 'ok' at the end`
			`of the program if none of the assert()s fired.`

			`This is wrong. They should do the test outside of an assert(), and print`
			`'ok' or 'not ok' as appropriate, instead of bailing out part way through.`
			`That way if a test starts failing we'll know exactly which one failed, and`
			`the other tests can continue.`

			`This probably means a simple libtest.so that they can link against to`
			`provide an ok() function (the non-fatal assert), probably with some helper`
			`functions to keep track of test numbers and so on.`

			`---`

			`Any of the tests written in C should link against the libraries under`
			`/usr/obj rather than using the system libraries.`

			`---`

			`Not everything's been converted to the new style. In particular;`

			`atm/`
			`ccd/`
			`some of geom_*/`
			`some of netinet/`
			`nfsmmap/`
			`p1003_1b/`
			`some of pipe/`
			`security/`
			`some of sockets/`
			`tls/`

			`needs to be converted. And fsx/ and gaithrstress/ are (I think) standalone`
			`programs rather than individual tests, so aren't amenable to this sort of`
			`conversion.`