Dane's Stuff

Test Properties


In a previous post I talked about testable code. Kent Beck expanded on this a bit with (Test Desiderata) [https://kentbeck.github.io/TestDesiderata/] and talks about properties of good tests. I really like how he talks about combinations and how certain properties can interfere with each other so we have to make trade-offs.

Lets talk about some of the trade-offs with unit tests. There is a mnemonic of (FIRST)[https://medium.com/pragmatic-programmers/unit-tests-are-first-fast-isolated-repeatable-self-verifying-and-timely-a83e8070698e] that states what a unit test should be. These map really well to some of Beck's test properties, but not all of them. It's important to remember that different types of tests have strengths and weaknesses and we should lean into them and use the type that fits what we are trying to accomplish.

Unit tests should be fast, repeatable, deterministic, and easily ran as part of the development. They are more useful for complex pieces of code, especially pieces with lots of variation in inputs. They help find edge cases and refactor code so that it is maintainable. They don't give much feedback on the correctness of the application, they are not very predictive, which is what lots of people mistakenly think they do. However, you get more feedback on the design of a particular section of code and whether changes broke other parts of the code.

However, if we want to ensure a end user can use the UI for some critical functionality, then an end to end test would be more appropriate. We would trade some properties, like fast and specific for some other properties, such as predictive and inspiring.

So depending on what you are trying to do, and what kind of feedback you are trying to gain, determines what kind of tests. A application with mostly UI, might have more end to end and integration test, while one that was more complex calculations would be more focused on unit tests. I think this is a point lots of people miss. Not all tests do the same thing, each one has strengths and weaknesses, and knowing what problem you are trying to solve will allow you to choose the correct test.