Effective Unit Testing With Hypothesis and pytest

Unit testing is a crucial aspect of software development, ensuring that individual components of a system work as expected. While traditional unit testing involves manually crafting test cases, tools like Hypothesis and pytest can significantly streamline and enhance the testing process.

Introduction to Hypothesis

Hypothesis is a powerful library for property-based testing in Python. Unlike traditional unit testing, where you explicitly define test cases, Hypothesis lets you specify the properties or invariants your code should satisfy. It then automatically generates a diverse set of test cases to validate those properties.

One of the key advantages of Hypothesis is its ability to generate edge cases and corner cases that may be overlooked when manually writing tests. This greatly improves the robustness and reliability of your code.

Setting up Hypothesis with pytest

As you all know, I’m a fan of pytest, the popular testing framework for Python that provides a simple and intuitive way to write and run tests. Combining Hypothesis with pytest lets you leverage the strengths of both.

Install both libraries:

pip install pytest hypothesis

Import the necessary modules in your test file:

from hypothesis import given
import hypothesis.strategies as st

Writing property-based tests with Hypothesis

Let’s consider a simple example of testing a function that calculates the sum of two numbers:

def add(x, y):
    return x + y

With traditional unit testing, you might write tests like:

def test_add():
    assert add(2, 3) == 5
    assert add(0, 0) == 0
    assert add(-1, 1) == 0

These cover some basic scenarios, but may miss edge cases or unexpected inputs.

With Hypothesis, you define the property that add should satisfy:

@given(st.integers(), st.integers())
def test_add_is_commutative(x, y):
    assert add(x, y) == x + y

The @given decorator tells Hypothesis to generate test cases using the provided strategies (st.integers() for both x and y). Hypothesis automatically generates a wide range of integer values, ensuring add is thoroughly tested.

You can further customize the strategies to explore specific scenarios:

@given(
    st.integers(min_value=-1000, max_value=1000),
    st.integers(min_value=-1000, max_value=1000),
)
def test_add_with_bounds(x, y):
    assert add(x, y) == x + y

This generates integers between -1000 and 1000 for both arguments.

Running tests with pytest

If you already have pytest, you just need a module named test_xyz.py and it’ll automatically pick up all your tests:

from hypothesis import given
import hypothesis.strategies as st


def add(x, y):
    return x + y


@given(st.integers(), st.integers())
def test_add_is_commutative(x, y):
    assert add(x, y) == x + y


@given(
    st.integers(min_value=-1000, max_value=1000),
    st.integers(min_value=-1000, max_value=1000),
)
def test_add_with_bounds(x, y):
    assert add(x, y) == x + y

When we run it, the output is:

============================= test session starts ==============================
collected 2 items

test_example.py::test_add_is_commutative PASSED [ 50%]
test_example.py::test_add_with_bounds PASSED [100%]

============================== 2 passed in 0.38s ===============================

Now let’s introduce a bug:

def add(x, y):
    return x - y  # Incorrect implementation

Running the tests:

test_example.py::test_add_is_commutative FAILED [ 50%]
-1 != 1
Expected: 1
Actual: -1

test_example.py::test_add_with_bounds FAILED [100%]
-1 != 1
Expected: 1
Actual: -1

Hypothesis effectively generates test cases that expose issues in your code, while pytest provides a convenient way to run and report on them.

Debugging and looking at the code itself

While it’s easy to explain how it happens in theory, in reality it’s much cooler to look into the bits and bytes.

In the attached example, I started debugging the code to show how Hypothesis generates the tests with the given strategy. It fetches integers and randomly injects them into the test case skeleton, making it super reusable.

There are other ways to parametrize tests (the built-in pytest generator), but for such cases I personally go with Hypothesis.

Conclusion

Combining Hypothesis and pytest offers a powerful approach to unit testing in Python. Hypothesis lets you define properties and automatically generates test cases, ensuring thorough coverage and catching edge cases that might be missed with traditional unit testing. pytest provides a user-friendly testing framework, making it easy to run and manage your tests.

By leveraging these tools, you write more robust and reliable code, catch bugs earlier, and ultimately deliver higher-quality software.