There is more nuance in the ways people expect to run tests, as such this still has some breakage. Revert for now.