The test pyramid – what, when and how to test to get profits

Wojciech Gola | Quality Assurance | 05.07.2021

What is a testing pyramid?

According to the ISTQB dictionary, a test pyramid is “a graphical model representing the relationship of the amount of testing per level, with more at the bottom than at the top.”

Simply put, the test pyramid shows us the right proportions among different types of tests, where the tests that are faster to perform, more stable, and therefore cheaper should be a foundation of the pyramid. Tests that take more time to prepare and run, are less stable and involve more human work should be on the higher levels of the pyramid.

Practical test pyramid

For newbie testers who take the first steps in software development project, testing pyramid is like a roadmap that defines test suite, clarifies how many tests should be performed and gives other tips for writing automated tests. But the clue of testing pyramid is return of investment corresponding with automation of tests. More tests on lower levels equal higher ROI. So the aim of test pyramid is to create more reliable test suite.

Modern software development organisations use testing strategy that defines test coverage and sets up a reliable test suite. The testing pyramid is a framework that illustrates testing strategy for the specific software development project. It grounds for fewer tests and facilitates increasing the efficiency in such techniques as Test-Driven Development.

Test levels

In a general approach, the testing pyramid framework contains three main groups of tests. At the lower level, there are unit tests; integration tests are in the middle, and End-to-End tests at the top.

Unit tests

Unit tests test single or the smallest elements, e.g. methods, or classes in the program. Most often, they are created by developers at the stage of writing subsequent sections of code. Unit tests are quick to run and they are the most common type of test (for example, when building a project). They are very important in the project as they are the best way to provide immediate feedback on whether a given function is returning the correct result. Therefore, most tests are unit tests, and they create the foundation of the pyramid.

Unit tests are some of the simplest test classes of automated tests. To write tests for unit testing suite, there is no need to build sophisticated logic. Unit tests could be described with the use of the following advantages:

• check particular elements,
• fast feedback,
• easy to maintain,
• low maintenance costs.

Integration tests

Integration tests check whether individual modules, services or systems work properly together. They test the interfaces and interactions between these elements. Conducting tests of each integration point is, however, more demanding and therefore expensive – it requires running several application modules or even systems simultaneously.

Integration test is a more advanced test type than a unit test but is necessary for a whole test suite. As it is more complicated, integration testing involves more resources and knowledge to provide good quality of such service tests. Integration tests are characterized by the following features:

• they check several modules or systems
• more time-consuming than unit tests
• higher maintenance costs than unit tests

E2E tests

E2E tests imitate the behavior of the end user. The tester takes on the role of the end user and follows the same paths that they could follow. These tests check the end-to-end functionality and are performed in the target production environment. They check whether all systems and their components work correctly together and are free of errors. E2E tests are the most time-consuming, and therefore the most expensive, which is why they are at the very top of the pyramid.

• they check the overall functioning of the system
• time-consuming
• more difficult and expensive to maintain

The cost of software testing – how to test and not to overpay

What is the aforementioned cost of software testing? In a nutshell, time is money, and the more people get involved in testing, and the more time spent on it, the higher the total cost of the project. Moreover, the later errors are detected (as part of subsequent levels of testing), the later the developer will receive information about it, and the later he will get to work on fixing them.

For example, the causes of defects reported only during  E2E testing are much more difficult to identify than those detected at the unit testing stage. This of course ultimately lengthens the overall timespan, and as a result, the cost of the project increases. By using the test pyramid method, the entire software testing process begins at the early stages of application development and testing is most frequent at these stages. Bugs found during unit testing are very easy to find, which expedites fixing them. Such an approach means that tests not only incur costs, but also bring real profits.

Testing strategy frameworks alternatives

For many different software development approaches (e.g. Continuous Integration – Continuous Delivery) there are several types of alternative frameworks including ice cream cone, the agile testing pyramid or test trophy.

External services

To ensure the best ROI, different teams use various testing strategies. One of the possible approaches to build a well-working test suite is to use external services. Outsourcing companies offer a vast range of contract tests in which flexible and on-demand cooperation models are some of the biggest benefits.

Test pyramid model – summary

Hiring a software tester involves an additional budget – but this way we invest in the project on our own terms defined in the contract. In the event of a failure resulting from the late detection of defects in the developed system, the system itself becomes the master of our budget (and time) and we bear unforeseen repair costs, that can be even several dozen times higher than the costs of testing. In order for the testing to bring profits, it is worth starting software testing as soon as possible – the test pyramid method is priceless here, as it helps to properly plan tests at particular levels.

Read more: Common myths related to test automation