Introduction

Godkjenn is a test for performing approval testing in Python. Approval testing is a simple yet powerful technique for testing systems that may not be amenable to more traditional kinds of tests or for which more traditional tests might not be particularly valuable. Approval testing can also be extremely useful for adding tests to legacy systems where simply capturing the behavior of the system as a whole is the first step in adding more sophisticated tests.

Principle

The principle of approval testing simple. Given a function or program that you consider correct, you store its output for a given input. This is the accepted or golden version of its output. Then, as you change your code, you reproduce the output (we call this received output) and compare it to the accepted version. If they match, then the test passes. Otherwise, it fails.

A test failure can mean one of two things. First, it could mean that you actually broke you program and need to fix it so that the received output matches the accepted. Second, it could mean the received output is now correct, the accepted output is now out of date, and you need to update the accepted output with the received.

As an approval testing tool, godkjenn aims to streamline and simplify this kind of testing.

Core elements

There are a few core elements to godkjenn. These are the parts of the approval testing system that are independent of any particular testing framework. Generally speaking, you won’t need to work with these directly; the integration with your testing framework with hide most of the low-level details.

Vaults

Vaults are where the accepted outputs are stored. (The term vault is a bit of a play on words: the accepted output is “golden”, and you keep gold in vaults.)

The vault abstraction defines an API for storing and retrieving accepted (and received) output.

godkjenn provides a simple vault, FSVault, that stores its data on the filesystem. Other vaults can be provided via a plugin system.

Verification

The core verification algorithm compares new received data with the accepted data for a given test. If there’s a mismatch or if not accepted output exists, this triggers test failure and instructs the user on what to do next.

Diffing

When an approval test fails, godkjenn provides tools for viewing the differences between the accepted and received data. godkjenn comes with some very basic fallback diffing support, and it provides a way to run external diffing tools. You can even have it use different tools for different types of files.