Last Updated on March 15, 2022
Unit testing is a method for testing software which looks at the smallest testable pieces of code, called units, are tested for correct operation. By doing unit testing, we are able to verify that each part of the code, including helper functions which may not be exposed to the user, works correctly and as intended.
The idea is that we are independently checking each small piece of our program to ensure that it works. This contrasts with regression and integration testing which tests that the different parts of the program work well together and as intended.
In this post, you will discover how to implement unit testing in Python using two popular unit testing frameworks, the built-in PyUnit framework and the PyTest framework.
After completing this tutorial, you will know:
Unit testing libraries in Python such as PyUnit and PyTest
Checking expected function behavior through the use of unit tests
Let’s get started!
A Gentle Introduction to Unit Testing in Python
Photo by Bee Naturalles. Some rights reserved.
Overview
The tutorial is divided into five parts, they are:
What are unit tests and why are they important?
What is Test Driven Development (TDD)?
Using Python’s built-in PyUnit framework
Using PyTest library
Unit testing in action
What are unit tests and why are they important?
Remember doing math back in school, completing different arithmetic procedures before combining them to get the correct answer? Imagine how you would check to ensure that the calculations done at each step were correct and you didn’t make any careless mistakes or miswrote anything.
Now, extend that idea to code! We wouldn’t want to have to constantly look through our code to statically verify its correctness, so how would you create a test to ensure that the following piece of code actually returns the area of the rectangle?
def calculate_area_rectangle(width, height):
return width * height
We could run the code with a few test examples and see if it returns the expected output.
That’s the idea of a unit test! A unit test is a test which checks a single component of code, usually modularized as a function, and ensuring that it performs as expected.
Unit tests are an important part of regression testing to ensure that the code still functions as expected after we have made changes to the code and helps to ensure code stability. After making changes to our code, we can run the unit tests we have created previously to ensure that the existing functionality in other parts of the codebase has not been impacted by our changes.
Another key benefit of unit tests is that they help to easily isolate errors. Imagine running the entire project and receiving a string of errors. How would we go about debugging our code?
That’s where unit tests come in, we can analyze the outputs of our unit tests to see if any component of our code has been throwing errors and start debugging from there. That’s not to say that unit testing can always help us find the bug, but it allows for a much more convenient starting point before we start looking at the integration of components in integration testing.
For the rest of the article, we will be showing how to do unit testing by testing the functions in this Rectangle class
class Rectangle:
def __init__(self, width, height):
self.width = width
self.height = height
def get_area(self):
return self.width * self.height
def set_width(self, width):
self.width = width
def set_height(self, height):
self.height = height
Now that we have motivated unit tests, let’s explore how exactly we can use unit tests as part of our development pipeline and how to implement them in Python!
Test Driven Development
Testing is so important to good software development that there’s even a software development process based on testing, which is Test Driven Development (TDD). Three rules of TDD proposed by Robert C. Martin are:
You are not allowed to write any production code unless it is to make a failing unit test pass.
You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
The key idea of TDD is that we base our software development around a set of unit tests that we have created, which makes unit testing the heart of the TDD software development process. In this way, you assured that you have a test for every component you developed.
TDD also biases towards having smaller tests which means tests that are more specific and test fewer components at a time. This aids in tracking down errors and smaller tests are also easier to read and understand since there are less components at play in a single run.
It doesn’t mean you must use TDD for your projects. But you may consider that as a method to develop your code and the tests at the same time.
Using Python built-in PyUnit framework
You might be wondering, why do we need unit testing frameworks since Python and other languages offer the assert keyword? Unit testing frameworks help to automate the testing process and allow us to run multiple tests on the same function with different parameters, check for expected exceptions, and many others.
PyUnit is Python’s built-in unit testing framework and is Python’s version of the corresponding JUnit testing framework for Java. To get started building a test file, we need to import the unittest library to use PyUnit:
import unittest
Then, we can get started writing out first unit test. Unit tests in PyUnit are structured as subclasses of the unittest.TestCase class and we can override the runTest() method to perform our own unit tests which check conditions using different assert functions in unittest.TestCase:
class TestGetAreaRectangle(unittest.TestCase):
def runTest(self):
rectangle = Rectangle(2, 3)
self.assertEqual(rectangle.get_area(), 6, “incorrect area”)
That’s our first unit test! It checks if the rectangle.get_area() method returns the correct area for a rectangle with width = 2 and length = 3. We use self.assertEqual instead of simply using assert to allow the unittest library to allow the runner to accumulate all test cases and produce a report.
Using the different assert functions in unittest.TestCase also gives us a better ability to test different behaviors such as self.assertRaises(exception) which allows us to check if a certain block of code produces an expected exception.
To run the unit test, we make a call to unittest.main() in our program,
…
unittest.main()
Since the code returns the expected output for this case, it returns that the tests run successfully, with the output:
.
———————————————————————-
Ran 1 test in 0.003s
OK
The complete code is as follows:
import unittest
# Our code to be tested
class Rectangle:
def __init__(self, width, height):
self.width = width
self.height = height
def get_area(self):
return self.width * self.height
def set_width(self, width):
self.width = width
def set_height(self, height):
self.height = height
# The test based on unittest module
class TestGetAreaRectangle(unittest.TestCase):
def runTest(self):
rectangle = Rectangle(2, 3)
self.assertEqual(rectangle.get_area(), 6, “incorrect area”)
# run the test
unittest.main()
Note: While in the above, our business logic Rectangle class and our test code TestGetAreaRectangle are put together, in reality you may put them in separate files and import the business logic into your test code. This can help you better manage the code.
We can also nest multiple unit tests together in one subclass of unittest.TestCase, by naming methods in the new subclass with the “test” prefix, for example:
class TestGetAreaRectangle(unittest.TestCase):
def test_normal_case(self):
rectangle = Rectangle(2, 3)
self.assertEqual(rectangle.get_area(), 6, “incorrect area”)
def test_negative_case(self):
“””expect -1 as output to denote error when looking at negative area”””
rectangle = Rectangle(-1, 2)
self.assertEqual(rectangle.get_area(), -1, “incorrect negative output”)
Running this will give us our first error:
F.
======================================================================
FAIL: test_negative_case (__main__.TestGetAreaRectangle)
expect -1 as output to denote error when looking at negative area
———————————————————————-
Traceback (most recent call last):
File “<ipython-input-96-59b1047bb08a>”, line 9, in test_negative_case
self.assertEqual(rectangle.get_area(), -1, “incorrect negative output”)
AssertionError: -2 != -1 : incorrect negative output
———————————————————————-
Ran 2 tests in 0.003s
FAILED (failures=1)
We can see the unit test which failed, which is the test_negative_case as highlighted in the output along with the stderr message, since get_area() doesn’t return -1 as we expected in our test.
If we had some code that we needed to run to set up before running each test, we can override the setUp() method:
class TestGetAreaRectangleWithSetUp(unittest.TestCase):
def setUp(self):
self.rectangle = Rectangle(0, 0)
def test_normal_case(self):
self.rectangle.set_width(2)
self.rectangle.set_height(3)
self.assertEqual(self.rectangle.get_area(), 6, “incorrect area”)
def test_negative_case(self):
“””expect -1 as output to denote error when looking at negative area”””
self.rectangle.set_width(-1)
self.rectangle.set_height(2)
self.assertEqual(self.rectangle.get_area(), -1, “incorrect negative output”)
In the above code example, we have overridden the setUp() method from unittest.TestCase, with our own setUp() method that initializes a Rectangle object. This setUp() method is run prior to each unit test and is helpful in avoiding code duplication when multiple tests rely on the same piece of code to set up the test. This is similar to the @Before decorator in JUnit.
Likewise, there is a tearDown() method that we can override as well for code to be executed after each test.
We can also aggregate test cases into test suites which help to group tests that should executed together into a single object:
…
# loads all unit tests from TestGetAreaRectangle into a test suite
calculate_area_suite = unittest.TestLoader()
.loadTestsFromTestCase(TestGetAreaRectangleWithSetUp)
Here, we also introduce another way to run tests in PyUnit which is using the unittest.TextTestRunner class, which allows us to run specific test suites.
runner = unittest.TextTestRunner()
runner.run(calculate_area_suite)
which gives the same output as running the file from the command line and calling unittest.main().
The complete code is as follows:
import unittest
# Our code to be tested
class Rectangle:
def __init__(self, width, height):
self.width = width
self.height = height
def get_area(self):
return self.width * self.height
def set_width(self, width):
self.width = width
def set_height(self, height):
self.height = height
# The test based on unittest module
class TestGetAreaRectangleWithSetUp(unittest.TestCase):
def setUp(self):
self.rectangle = Rectangle(0, 0)
def test_normal_case(self):
self.rectangle.set_width(2)
self.rectangle.set_height(3)
self.assertEqual(self.rectangle.get_area(), 6, “incorrect area”)
def test_negative_case(self):
“””expect -1 as output to denote error when looking at negative area”””
self.rectangle.set_width(-1)
self.rectangle.set_height(2)
self.assertEqual(self.rectangle.get_area(), -1, “incorrect negative output”)
# run the test
calculate_area_suite = unittest.TestLoader()
.loadTestsFromTestCase(TestGetAreaRectangleWithSetUp)
runner = unittest.TextTestRunner()
runner.run(calculate_area_suite)
This is just the tip of the iceberg with what you can do with PyUnit. We can also write tests that look for exception messages that match a regex expression or setUp/tearDown methods that are run only once (setUpClass), for example.
Using PyTest
PyTest is an alternative to the built-in unittest module. To get started with PyTest, you will first need to install it, which you can do using
pip install pytest
To write tests, you just need to write functions with names prefixed with “test” and PyTest’s test discovery procedure will be able to find your tests, e.g.,
def test_normal_case(self):
rectangle = Rectangle(2, 3)
assert rectangle.get_area() == 6, “incorrect area”
You will notice that PyTest uses Python’s built-in assert keyword instead of its own set of assert functions as PyUnit does, which might make it slightly more convenient since we can avoid searching up the different assert functions.
The complete code is as follows:
# Our code to be tested
class Rectangle:
def __init__(self, width, height):
self.width = width
self.height = height
def get_area(self):
return self.width * self.height
def set_width(self, width):
self.width = width
def set_height(self, height):
self.height = height
# The test function to be executed by PyTest
def test_normal_case():
rectangle = Rectangle(2, 3)
assert rectangle.get_area() == 6, “incorrect area”
After saving this into a file test_file.py, we can run PyTest unit test by:
python -m pytest test_file.py
And this gives us the output:
=================== test session starts ====================
platform darwin — Python 3.9.9, pytest-7.0.1, pluggy-1.0.0
rootdir: /Users/MLM
plugins: anyio-3.4.0, typeguard-2.13.2
collected 1 item
test_file.py . [100%]
==================== 1 passed in 0.01s =====================
You may notice that while in PyUnit, we need to invoke the test routine by a runner or calling unittest.main(). But in PyTest, we simply pass the file to the module. The PyTest module will collect all the functions defined with prefix test and call them one by one. And then it will verify if any exception is raised by the assert statement. It can be more convenient to allow the tests stay with the business logic.
PyTest also supports grouping functions together in classes, but the class should be named with prefix “Test” (with uppercase T), e.g.
class TestGetAreaRectangle:
def test_normal_case(self):
rectangle = Rectangle(2, 3)
assert rectangle.get_area() == 6, “incorrect area”
def test_negative_case(self):
“””expect -1 as output to denote error when looking at negative area”””
rectangle = Rectangle(-1, 2)
assert rectangle.get_area() == -1, “incorrect negative output”
Running this with PyTest will produce the following output:
=================== test session starts ====================
platform darwin — Python 3.9.9, pytest-7.0.1, pluggy-1.0.0
rootdir: /Users/MLM
plugins: anyio-3.4.0, typeguard-2.13.2
collected 2 items
test_code.py .F [100%]
========================= FAILURES =========================
_________ TestGetAreaRectangle.test_negative_case __________
self = <test_code.TestGetAreaRectangle object at 0x10f5b3fd0>
def test_negative_case(self):
“””expect -1 as output to denote error when looking at negative area”””
rectangle = Rectangle(-1, 2)
> assert rectangle.get_area() == -1, “incorrect negative output”
E AssertionError: incorrect negative output
E assert -2 == -1
E + where -2 = <bound method Rectangle.get_area of <test_code.Rectangle object at 0x10f5b3df0>>()
E + where <bound method Rectangle.get_area of <test_code.Rectangle object at 0x10f5b3df0>> = <test_code.Rectangle object at 0x10f5b3df0>.get_area
unittest5.py:24: AssertionError
================= short test summary info ==================
FAILED test_code.py::TestGetAreaRectangle::test_negative_case
=============== 1 failed, 1 passed in 0.12s ================
The complete code is as follows:
# Our code to be tested
class Rectangle:
def __init__(self, width, height):
self.width = width
self.height = height
def get_area(self):
return self.width * self.height
def set_width(self, width):
self.width = width
def set_height(self, height):
self.height = height
# The test functions to be executed by PyTest
class TestGetAreaRectangle:
def test_normal_case(self):
rectangle = Rectangle(2, 3)
assert rectangle.get_area() == 6, “incorrect area”
def test_negative_case(self):
“””expect -1 as output to denote error when looking at negative area”””
rectangle = Rectangle(-1, 2)
assert rectangle.get_area() == -1, “incorrect negative output”
To implement setup and teardown code for our tests, PyTest has an extremely flexible fixture system, where fixtures are functions that have a return value. PyTest’s fixture system allows for sharing of fixtures across classes, modules, packages, or sessions, and fixtures that can call other fixtures as arguments.
Here we include a simple introduction to PyTest’s fixture system,
@pytest.fixture
def rectangle():
return Rectangle(0, 0)
def test_negative_case(rectangle):
print (rectangle.width)
rectangle.set_width(-1)
rectangle.set_height(2)
assert rectangle.get_area() == -1, “incorrect negative output”
The above code introduces rectangle as a fixture and PyTest matches the rectangle in the argument list of test_negative_case with the fixture and provides test_negative_case with its own set of outputs from the rectangle function, and does this for every other test. However, note that fixtures can be requested more than once per test and for each test, the fixture is only run once and the result is cached. This means that all references to that fixture during the running of an individual test are referencing the same return value (which is important if the return value is a reference type).
The complete code is as follows:
import pytest
# Our code to be tested
class Rectangle:
def __init__(self, width, height):
self.width = width
self.height = height
def get_area(self):
return self.width * self.height
def set_width(self, width):
self.width = width
def set_height(self, height):
self.height = height
@pytest.fixture
def rectangle():
return Rectangle(0, 0)
def test_negative_case(rectangle):
print (rectangle.width)
rectangle.set_width(-1)
rectangle.set_height(2)
assert rectangle.get_area() == -1, “incorrect negative output”
Like PyUnit, PyTest has a lot of other functionality that will allow you to build more comprehensive and advanced unit tests.
Unit Testing in Action
Now, we’ll explore unit testing in action. For our example, we’ll be testing a function that gets stock data from Yahoo Finance using pandas_datareader and doing this in PyUnit:
import pandas_datareader.data as web
def get_stock_data(ticker):
“””pull data from stooq”””
df = web.DataReader(ticker, “yahoo”)
return df
This function gets the stock data on a particular stock ticker by crawling from Yahoo Finance website and returns the pandas DataFrame. This can fail in multiple ways, for example, the data reader may failed to return anything (if Yahoo Finance is down), or return a DataFrame with missing columns or missing data in the columns (if the source restructured its website). Therefore, we should provide multiple test functions to check for multiple modes of failure:
import datetime
import unittest
import pandas as pd
import pandas_datareader.data as web
def get_stock_data(ticker):
“””pull data from stooq”””
df = web.DataReader(ticker, ‘yahoo’)
return df
class TestGetStockData(unittest.TestCase):
@classmethod
def setUpClass(self):
“””We only want to pull this data once for each TestCase since it is an expensive operation”””
self.df = get_stock_data(‘^DJI’)
def test_columns_present(self):
“””ensures that the expected columns are all present”””
self.assertIn(“Open”, self.df.columns)
self.assertIn(“High”, self.df.columns)
self.assertIn(“Low”, self.df.columns)
self.assertIn(“Close”, self.df.columns)
self.assertIn(“Volume”, self.df.columns)
def test_non_empty(self):
“””ensures that there is more than one row of data”””
self.assertNotEqual(len(self.df.index), 0)
def test_high_low(self):
“””ensure high and low are the highest and lowest in the same row”””
ohlc = self.df[[“Open”,”High”,”Low”,”Close”]]
highest = ohlc.max(axis=1)
lowest = ohlc.min(axis=1)
self.assertTrue(ohlc.le(highest, axis=0).all(axis=None))
self.assertTrue(ohlc.ge(lowest, axis=0).all(axis=None))
def test_most_recent_within_week(self):
“””most recent data was collected within the last week”””
most_recent_date = pd.to_datetime(self.df.index[-1])
self.assertLessEqual((datetime.datetime.today() – most_recent_date).days, 7)
unittest.main()
Our series of unit tests above check if certain columns are present (test_columns_present), whether the dataframe is non-empty (test_non_empty), whether the “high” and “low” columns are really the high and low of the same row (test_high_low), and whether the most recent data in the DataFrame was within the last 7 days (test_most_recent_within_week).
Imagine you are doing a machine learning project that consumes the stock market data. Having a unit test framework can help you identify if your data preprocessing is working as expected.
Using these unit tests, we are able to identify if there was a material change in the output of our function, and can be a part of a Continuous Integration (CI) process. We can also attach other unit tests as required depending on the functionality that we depend on from that function.
If you like it then you should have put a test on it
— Software Engineering at Google
Further reading
This section provides more resources on the topic if you are looking to go deeper.
Libraries
unittest module (and the list of assert methods), https://docs.python.org/3/library/unittest.html
PyTest, https://docs.pytest.org/en/7.0.x/
Articles
Test Driven Development (TDD), https://www.ibm.com/garage/method/practices/code/practice_test_driven_development/
Python Unit Testing Framework, http://pyunit.sourceforge.net/pyunit.html
Books
Software Engineering at Google, by Titus Winters, Tom Manshreck, and Hyrum Wright https://www.amazon.com/dp/1492082791
Summary
In this post, you discovered what unit testing is and how to use two popular libraries in Python to conduct unit testing (PyUnit, PyTest). You have also learnt how to configure unit tests and seen an example of a use case for unit testing in the data science pipeline.
Specifically, you learned:
what unit testing is and why it is useful
how unit testing fits within the Test Driven Development pipeline
how to do unit testing in Python using PyUnit and PyTest
The post A Gentle Introduction to Unit Testing in Python appeared first on Machine Learning Mastery.
Read MoreMachine Learning Mastery
