🏆 This post was featured in Software Testing Weekly #173
tl;dr A visual metaphor that groups tests into layers and recommends how many tests should exist in each of layers.
This is part of my free testing course, focused on teaching you the fundamentals of testing 😉
Theory
The concept of a test pyramid was initially introduced by Mike Cohn. It is a visual metaphor that groups tests into layers and recommends how many tests we should have in each of these layers.
This visual model conveys several messages:
- You should have multiple test types or layers
- You should have more unit tests than service steps, and more of those than UI tests
- Tests at the base of the pyramid (unit) are faster, cheaper and more precise
- Tests at the top of the pyramid (UI) are slower, more expensive and more realistic
Unit tests ensure a small and specific unit of code works as intended in isolation. That’s why they are cheap and fast to run. These units of code then interact with other parts the system to provide services. The behaviour of those services is tested by the service tests. Finally, a human needs to interact with those services and that is done through a User Interface (UI). The UI tests check that the system as a whole does what the user expects, and that’s why those tests are the slowest to run and most expensive to debug.
The test pyramid is a way of thinking about how different kinds of automated tests should be used to create a balanced portfolio
Since it was introduced in 2009, new test types were developed. The “service tests” are nowadays called “integration tests” and may include API testing and Contract testing. The “UI tests” are nowadays split between frontend testing and end-to-end testing.
The simplicity and usefulness of this model made the test pyramid very popular, and many engineers and testers use it as a reference still today.
“All models are wrong, some are useful”
Despite it’s popularity, it is not without flaws. Some people say the model is overly simplistic because it doesn’t include all testing done for a project. Others say it is biased towards automation, like Dan Ashby who calls it “the automation triangle”.
The testing trophy
I want to be confident that my code satisfies the requirements and I’ll use a mix of the different testing strategies to accomplish that goal.
Almost 10 years later, Kent Dodds published his modern take on the test pyramid and called it the testing trophy. It’s an overall improvement over the previous model in several aspects.
Here’s a summary, paraphrasing the author:
- Purpose. Why do you write tests? Confidence. I want to be confident that the code I’m writing won’t break the app. I want to get the most confidence out of the tests I write, and I’m aware of the trade-offs I’m making when testing.
- Static tests. Linters and other static analysis tools scan your code for typos, type errors, common mistakes, potential bugs.
- Unit tests: Verify that individual, isolated parts work as expected.
- Integration tests: Verify that several units work together in harmony.
- End-to-End tests: Simulate the user interacting with the app and verify that it functions correctly.
- Flexible. The size of each of test layer may differ based on what your team values. The proportions are not meant to be taken as rigid rules. It also depends on how easy it is to test your app with the tools available.
Every level comes with its own trade-offs. An E2E test has more points of failure making it often harder to track down the problem, but it also means that your test is giving you more confidence.
Practice
As with any tool, “keep the best and discard the rest”.
Think of the test pyramid as an heuristic to trigger your thinking, a starting point — it should not be blindly pursued as perfection or the end goal. It teaches you three valuable lessons:
- Write tests with different granularity
- High-level tests should be realistic (and you pay for it)
- Low-level tests should be fast and precise
- If a higher-level test fails without a lower-level test failing too, you need to write a lower-level test
That last one needs a bit more explanation. Tests at the top of the pyramid exercise multiple parts of the system at the same time. If one of those tests fail, it tells you “there’s a problem with X” but without a lower-level test you will not know the cause of the problem. And they will serve as a good regression test for the future.
It also keeps your test suite fast. If you have tested all conditions on a lower-level test, the extra confidence you get from a higher-level test is small. Redundant tests makes your development process more costly because you need to change more tests when you change the behaviour of your code.
The more your tests resemble the way your software is used, the more confidence they can give you.
The “pyramid” concept also suggests an analogy to construction. Each test layer is a different material used in the construction. A building made of a single material is not as strong as one that uses multiple specialised materials, each contributing differently to the stability of the building.
If you prefer cheese to construction, here’s another analogy. Swiss cheese slices have holes. We want to create a surface where you can’t see through, thus we layer one slice on top of the other. Individually, slices have holes (limitations) and don’t cover the whole surface, but together the surface of one layer covers the holes of the layer below!
And that’s what we want to achieve with our testing layers. Individually each layer has coverage gaps, but all together we efficiently maximise coverage. “The right tool for the right job”. “Divide to conquer”.
With time, many more models were created besides the original pyramid. Nowadays you have many flavours to pick from, just like ice creams. Speaking of which:
- the ice cream cone (or cupcake)
- the trophy
- the box of rocks
- the honeycomb
- the planet
- the snowman
- the wheel
- the filters
- and many more
What they all have in common is that their authors are trying to convey a message visually. Use whatever illustrates best your narrative or create your own.
