The PULL_BASE_SHA was sometimes being set as master which meant that if multiple commits came in, they'd all write their statuses to the newest one.
Since the needs of each project will be different, this seems like a better solution than relying on configs defined in my kubernetes repo.