Software testing costs arouse a great deal of excitement. Despite the fact that “testing in production” has become a synonym for negligence, you can still find those who don’t feel the need to employ software testers. Software testing can be an unnecessary cost – but only if tests are run too late, they are unstable, of poor quality, or planned inappropriately. How to properly choose the right proportion of individual tests at specific levels to avoid unnecessary costs related to repairing the developed software – as well as making sure that the tests are worthwhile? Here’s where the test pyramid comes in handy.

In this article you will learn:

  • What a testing pyramid is
  • How to schedule software tests
  • What unit tests, integration tests and E2E tests are
  • How to avoid unnecessary software development costs

What is a testing pyramid?

According to the ISTQB dictionary, a test pyramid is “a graphical model representing the relationship of the amount of testing per level, with more at the bottom than at the top.”

Simply put, the test pyramid shows us the right proportions among different types of tests, where the tests that are faster to perform, more stable, and therefore cheaper should be a foundation of the pyramid. Tests that take more time to prepare and run, are less stable and involve more human work should be on the higher levels of the pyramid.

Test levels

Unit tests

Unit tests test single or the smallest elements, e.g. methods, or classes in the program. Most often, they are created by developers at the stage of writing subsequent sections of code. Unit tests are quick to run and they are the most common type of test (for example, when building a project). They are very important in the project as they are the fastest way to provide feedback on whether a given function is returning the correct result. Therefore, most tests are unit tests, and they create the foundation of the pyramid.

  • they check particular elements
  • fast
  • easy to maintain
  • low maintenance costs

Integration tests

Integration tests check whether individual modules, services or systems work properly together. They test the interfaces and interactions between these elements. Conducting such tests is, however, more demanding and therefore expensive – it requires running several application modules or even systems simultaneously.

  • they check several modules or systems
  • more time-consuming than unit tests
  • higher maintenance costs than unit tests

E2E tests

E2E tests imitate the behavior of the end user. The tester takes on the role of the end user and follows the same paths that they could follow. These tests check the end-to-end functionality and are performed in the target production environment. They check whether all systems and their components work correctly together and are free of errors. E2E tests are the most time-consuming, and therefore the most expensive, which is why they are at the very top of the pyramid.

  • they check the overall functioning of the system
  • time-consuming
  • more difficult and expensive to maintain

The cost of software testing – how to test and not to overpay

What is the aforementioned cost of software testing? In a nutshell, time is money, and the more people get involved in testing, and the more time spent on it, the higher the total cost of the project. Moreover, the later errors are detected (as part of subsequent levels of testing), the later the developer will receive information about it, and the later he will get to work on fixing them.

For example, the causes of defects reported only during  E2E testing are much more difficult to identify than those detected at the unit testing stage. This of course ultimately lengthens the overall timespan, and as a result, the cost of the project increases. By using the test pyramid method, the entire software testing process begins at the early stages of application development and testing is most frequent at these stages. Bugs found during unit testing are very easy to find, which expedites fixing them. Such an approach means that tests not only incur costs, but also bring real profits.

Test pyramid model – summary

Hiring a software tester involves an additional budget – but this way we invest in the project on our own terms defined in the contract. In the event of a failure resulting from the late detection of defects in the developed system, the system itself becomes the master of our budget (and time) and we bear unforeseen repair costs, that can be even several dozen times higher than the costs of testing. In order for the testing to bring profits, it is worth starting software testing as soon as possible – the test pyramid method is priceless here, as it helps to properly plan tests at particular levels.

Read more: Common myths related to test automation

Author:
Tester

Tester z 5-letnim doświadczeniem. Wojciech zaczynał od testów manualnych, aktualnie zajmuje się również automatyzacją. Pracował głównie przy projektach webowych oraz mobilnych. Po godzinach odkrywa tajniki druku 3d.

Comment

CONTACT US!

Would you like to learn more about the possibilities of cooperation? Do you have a question? Write to us!

    I hereby agree that JCommerce Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as my name, surname, e-mail address, telephone number and company name, for commercial purposes.
    I hereby agree that JCommerce Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as my name, surname, e-mail address, telephone number and company name, for marketing purposes.
    I hereby agree that JCommerce Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as my name, surname, e-mail address, telephone number and company name, for recruitment purposes.
    I hereby agree that JCommerce Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as my name, surname, e-mail address, telephone number and company name, for future recruitment purposes.
    I have been informed by JCommerce Sp. z o.o., 3 Ks. Piotra Sciegiennego St. 40-114 Katowice – the personal data controller – that: - The provision of the aforementioned personal data is voluntary but essential for commercial purposes if I have chosen a request for proposal, or recruitment purposes, if I have chosen the remaining options;
    - I have the right to access the content of my data, including to receive copies of it and correct it, delete it and limit the processing of it, as well as the right to transfer it, the right to object to the processing of it, and the right to withdraw my consent at any time. However, the withdrawal of my consent shall not affect the lawfulness of processing carried out on the basis of the consent in question prior to its withdrawal;
    - A statement of withdrawal of my consent to the processing of personal data should be submitted to the headquarters of JCommerce Sp. z o.o. or sent to the following e-mail address: zgody@jcommerce.pl. The withdrawal of consent to the processing of personal data shall result in the inability to fulfil the aforementioned processing purposes;
    - The personal data provided shall be shared by JCommerce Sp. z o.o. with the company’s authorised employees and individuals collaborating with JC under civil-law contracts, who are involved in the implementation of the purpose of the processing;
    - The data provided shall be processed on the basis of the relevant provisions of Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation), ‘GDPR’;
    - Should you have any questions regarding the protection of your personal data, please contact us by e-mail: odo@jcommerce.pl;
    - The personal data provided shall be processed for the purpose for which it was supplied, or until I express my objection in this regard. In the event of filing an objection, JCommerce Sp. z o.o. shall no longer process my personal data for the aforementioned purposes, unless it can demonstrate that there are valid and legally justified grounds overriding my interests, rights and freedoms or my data is necessary to establish, pursue or defend a claim, if any;
    - I have the right to file a complaint to the supervisory authority if I consider that the processing of the aforementioned personal data violates the provisions of the General Data Protection Regulation of 27 April 2016.
    I hereby agree that JCommerce Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as my name, surname, e-mail address, telephone number and company name, for commercial purposes.
    I hereby agree that JCommerce Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as my name, surname, e-mail address, telephone number and company name, for marketing purposes.
    I hereby agree that JCommerce Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as my name, surname, e-mail address, telephone number and company name, for recruitment purposes.
    I hereby agree that JCommerce Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as my name, surname, e-mail address, telephone number and company name, for future recruitment purposes.
    I have been informed by JCommerce Sp. z o.o., 3 Ks. Piotra Sciegiennego St. 40-114 Katowice – the personal data controller – that: - The provision of the aforementioned personal data is voluntary but essential for commercial purposes if I have chosen a request for proposal, or recruitment purposes, if I have chosen the remaining options;
    - I have the right to access the content of my data, including to receive copies of it and correct it, delete it and limit the processing of it, as well as the right to transfer it, the right to object to the processing of it, and the right to withdraw my consent at any time. However, the withdrawal of my consent shall not affect the lawfulness of processing carried out on the basis of the consent in question prior to its withdrawal;
    - A statement of withdrawal of my consent to the processing of personal data should be submitted to the headquarters of JCommerce Sp. z o.o. or sent to the following e-mail address: zgody@jcommerce.pl. The withdrawal of consent to the processing of personal data shall result in the inability to fulfil the aforementioned processing purposes;
    - The personal data provided shall be shared by JCommerce Sp. z o.o. with the company’s authorised employees and individuals collaborating with JC under civil-law contracts, who are involved in the implementation of the purpose of the processing;
    - The data provided shall be processed on the basis of the relevant provisions of Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation), ‘GDPR’;
    - Should you have any questions regarding the protection of your personal data, please contact us by e-mail: odo@jcommerce.pl;
    - The personal data provided shall be processed for the purpose for which it was supplied, or until I express my objection in this regard. In the event of filing an objection, JCommerce Sp. z o.o. shall no longer process my personal data for the aforementioned purposes, unless it can demonstrate that there are valid and legally justified grounds overriding my interests, rights and freedoms or my data is necessary to establish, pursue or defend a claim, if any;
    - I have the right to file a complaint to the supervisory authority if I consider that the processing of the aforementioned personal data violates the provisions of the General Data Protection Regulation of 27 April 2016.