Functional vs. Non-Functional Testing: Can You Have One Without the Other?

One, the division between non-functional and functional testing is not set in stone, and for some testing types, categorizing them is no easy feat. Two, both functional and non-functional testing are essential for the success of your software testing project, albeit in different ways. Today we are going to take a closer look at the difference between functional and non functional requirements, where these two types of testing stand in the software testing process, and how they influence the cost of testing.

What is functional testing?

According to the ISTQB, functional testing is a type of testing that checks the functionality of a component or a whole system. To put it simply, functionality testing helps us make sure that the specific function of a software product is working exactly as intended. If we compare a software product to a vehicle, a functional requirement would be its ability to travel from point A to point B. In other words, this is what the product is expected to do flawlessly every time you want to use it and the reason why you are using the product at all.

What is non-functional testing?

ISTQB defines non-functional testing as a type of testing that deals with the components of the system that are not related to its functionality. The scope of non-functional testing can be downright endless and very much depends on the specifics of the product. Non-functional requirements have more to do with the way the product works for the end user rather than the result that is expected in the end. To continue our car analogy, non-functional requirements can be anything from the maximum speed a vehicle can achieve to the number of steps it takes to find the necessary station on the car radio.

Why is there a distinction between functional and non-functional testing?

The distinction between functional and non-functional testing is there not because these two types of testing exist independently from each other. It’s also not there because it matters too much to the QA engineers. For most of them, what matters is the result of testing, which is a software product that is free of bugs, vulnerabilities, and inconsistencies.

The biggest reason why there is any distinction between functional and non-functional testing at all is the convenience of the client. With a clear approach to categories and groups, it’s easier for the clients to opt for the types of testing that are essential for their project and opt out of the ones that are not particularly relevant to their needs.

Functional vs non functional testing: key differences

Which types of functional testing and non functional testing are the most vital ones and why?

There are dozens of both functional and non-functional testing types, and each one of them is important in its own right. We are not going to go out on a limb and say that some types of testing are more important for a project than others because there is no universal set of tests applicable to every single project. Still, there are several types of functional and non-functional tests that are performed especially often and, when executed right, have a particularly noticeable effect on the quality of the final product. Here are some of those testing types.

Functional testing types

Functional testing techniques help the QA team understand whether the product conforms to the initial requirements set for it by the developers. There are multiple types of functional tests performed every day, but these are three of the most common ones.

• UI testing. In most cases, the graphical user interface is the only way for the audience of your product to interact with it. A faulty interface is a fatal obstacle that prevents the users from accessing certain features of the product or even using the product altogether. There are dozens of different UI aspects that need to be tested, from functional validation to spelling.
• Unit testing. With the help of unit testing, a QA engineer can get as precise as possible when testing a newly created or recently updated software product. Unit testing deals with the smallest fragments of code and can be both manual for maximum precision or automated for maximum efficiency.
• Security testing. Security in a software system has always been important, but its importance reached an all-time high with the start of the COVID-19 pandemic. With stakes at an all-time high, both in terms of data safety and brand loyalty, companies cannot afford any security risks. There are companies that specialize in security testing, but it can also be done as part of a functional testing package.

Non functional testing types

Non-functional testing deals with the way the system operates as a whole, not the specific functions it contains. Every non-functional aspect of the system is important in its own right, but there are also some types of non-functional testing that are appropriate for any software project:

• UX testing. This type of testing helps the developers understand whether the product is easy for the end user to operate. UX testing is often mentioned together with UI testing, but they operate on different levels of the user story. While UI testing makes sure that all aspects of the interface are there and work properly, UX testing ensures that they all make for a satisfying user journey.

• Performance testing. Performance testing is both a type of testing, which analyzes the application size, speed, and reliability of a software product, and a group of functional tests, which may include stress testing, load testing, configuration testing, spike testing, scalability testing, and several other types of testing. These non-functional tests ensure stable performance under any circumstances.
• Compatibility testing. Given the variety of platforms your software product can be used on, compatibility testing is one of the most essential types of non functional testing to consider for a project. The QA team will test your application against different types of hardware, software, mobile devices, operating systems and their versions, browsers, and networks to ensure consistent performances throughout them all.

Types of testing that are sometimes overlooked but are still very important

In our experience as a software testing company, there are two primary types of clients who request QA services. The first type depends entirely on the expertise of their QA team at every stage of the process, including the choice of testing types and the scope of testing. The second type has a set of specific requirements and testing types that need to be done. However, in the second case, there is often a risk of some essential types of testing missing from the test plan. These are the three types of testing that are sometimes overlooked but don’t deserve to be:

• Installation testing. A lot of attention is paid to the way the user interacts with the product, but it starts way before the application is launched for the first time. To be precise, it starts when the application or another type of software product is installed on the user’s device. And the fewer obstacles the user will encounter along the way, the more satisfied they are going to be with the product.
• Localization testing. It is very rare that a software product, such as a website or an application, is intended for use only within a single country or region. The more parts of the world you are going to cover with the release of your product, the more you need to care about proper localization. It involves anything from the correct time and date format and currency settings to complying with each state’s laws and regulations.
• Integration testing. In the hierarchy of system testing, unit testing comes first, then comes integration testing, and system testing comes last. No one is going to overlook unit testing to test the individual components of the product or system testing to see how they all interact with each other, but it’s integration testing, which deals with software modules pulled into groups, that is sometimes mistakenly treated as an afterthought.

Functional testing vs non functional testing: where do they stand in the testing process?

Creating a detailed test plan and designing a testing process is one of the ways to ensure that the whole testing endeavor goes over smoothly and that no functional and non-functional aspect of the system goes untested. But where do functional and non-functional testing stand in relation to the testing process and is there a difference between functional and non functional testing from this particular point of view?

One of the key things to know about the testing process in general is that there is no universal approach that fits every project and scenario. In every new testing project, the engineers take into account dozens of different factors, from the scale of the project to its target audience.

In our 20+ years of experience as a software testing company, we begin every testing project by reviewing the documentation that came along with the code. The documentation alone should tell us enough to help us design the testing process correctly and make sure that it’s not missing anything important.

However, in some cases, the documentation is missing or is insufficient. In that case, our QA engineers will perform exploratory testing first with the help of exploratory testing tours. It is typically viewed as a standalone testing technique and is not included in the functional or non-functional testing categories. It helps the engineer map out the testing process and decide on both functional and non-functional requirements of the product that need to be tested. Alternatively, we can perform checklist-based testing when the software product has relatively standard specifications and architecture.

Once exploratory testing yields sizable results, the engineer is able to see the big picture and the specifics of the upcoming testing process. Typically, we will test the functionality of the product first and then move on to its non-functional aspects. However, that is also not the only correct way of doing things. It is possible to test some non-functional aspects of a product during functional testing or at the exploratory phase. For example, an experienced QA engineer can already spot usability and UX issues of a product while still defining the scope of testing.

Change-related testing and its importance

While the most common approach to categorizing software testing is dividing all testing types into functional and non-functional, many professional QA engineers, including those working at TestFort, now distinguish a third category of software testing — change-related tests.

As evidenced by their name, change-related software tests take place only after there have been any changes to the code. This includes smoke testing, sanity testing, regression testing, and retesting. From the perspective of the software testing process, change-related testing can take place at any stage, both after changes to functional and non-functional components of the product.

Performing change-related testing — most importantly, regression testing — can potentially delay the release of your product for another few weeks. However, it’s one of the most crucial aspects of a well-tested product. Change-related testing helps you make sure that neither functional nor non-functional aspects of your product have been negatively affected by the recent changes to the code and that no old bugs have reappeared without being detected by timely regression testing.

Functional and non-functional testing: the aspect of the cost

Whether you are developing a software project from scratch and will need to eventually test it or you are getting ready to release a recently completed product and view software testing as the final frontier between an unfinished and finished project, the aspect of cost is an important one to consider.

How do functional and non functional testing factor into the cost?

For most software product owners, the cost of the testing project is the next big concern after the quality of the testing services. With so many resources being poured into the development process, most developers and product owners cannot afford similar expenses when it comes to testing. This is why they often wonder about the effect of functional and non-functional testing on the overall cost of the testing project.

The important thing to understand here is that the budget for your testing project is not defined as much by the functional and non-functional tests that need to be performed as much by the cooperation model you’ve chosen. With the exception of a few standout types of testing, including security testing, UX testing, and performance testing, QA engineers don’t charge more for specific types of tests. That is why there shouldn’t be a distinction between functional and non-functional testing when it comes to budgeting your project.

Factors influencing the cost of testing

Besides the most obvious factors affecting the cost of the testing project, such as the cooperation model you decide on and the scale of the project, there are a few other factors that directly influence the overall price. These are the five of them that are featured in one way or another in every testing project.

• Size, seniority, and location of the team. The choice of a QA team is one of the most crucial aspects of setting up a new testing project, and that also includes its cost. The logic behind the team size and level of seniority is simple, but the location of the team is also very important. Hiring a QA team in Ukraine is going to cost you considerably less than the same team in the US or Western Europe.
• Functionality complexity. You don’t need to be a software engineer to understand that the more features an application has, the more resources it requires for thorough testing. Performing testing on an e-commerce website with thousands of products is always going to be more expensive than testing an application with a single core feature, such as an alarm clock.
• The complexity of UX and UI. Your product may have the most groundbreaking functionality, but it’s not going to be worth much when it’s paired with an interface that is impossible to navigate, riddled with bugs, or looks and feels outdated. Obviously, the more UI elements your product has and the more ways there are for a user to interact with the product, the more complex and expensive this part of testing is going to be.
• Amount of manual and automated testing. Both the amounts of manual and automated testing influence the cost of a software testing project, but in different ways. The consensus among many QA engineers is that while testing automation is more expensive to set up, it typically proves to be more cost-effective in the long run since the tests can be reused, whereas manual testing needs to be performed by hand, so its cost doesn’t change much as the project progresses.
• Amount of functional and non-functional testing. The scope of functional testing includes way more than testing the functionality of the product itself. The amount of unit testing, system testing, and user acceptance testing will affect the cost of the project. The same goes for non-functional testing. The more thorough you want to be with security, performance, compatibility, and other types of non-functional testing, the more your project is going to cost.

ROI of functional and non-functional testing

Return on investment is somewhat of a controversial topic in the software testing community. Some say that there is no definitive way to calculate ROI of software testing, while others go as far as to claim that ROI in software testing is a myth.

As it’s often the case, the truth is somewhere in the middle. Calculating the return on investment of your software testing efforts is incredibly difficult for one reason: you can know for sure how much your testing project costs, but you can’t know exactly how it influenced your profit.

On its own, software testing, neither functional nor non-functional, does not generate profit. But with the help of timely testing, you can increase your profit by preventing software bugs, security issues, usability problems, and other factors negatively affecting your product’s position in the market.

So is there a difference between functional and nonfunctional testing in terms of ROI? If we look at the place of these two types of testing in the testing process, we will see that functional testing often takes place earlier than non-functional testing. And we know from Boehm’s law that the earlier bugs are spotted, the fewer resources are spent on fixing them. Plus, functional bugs are often easier to locate, especially when there is enough documentation to begin with. All of this makes functional testing noticeably cost-effective.

Non-functional testing, on the other hand, typically happens at the final stages of the software product development or sometimes even after its release. At this point, fixing a bug takes significantly more time and money. Moreover, non-functional bugs are often harder to spot because they deal with the finer aspects of the software product’s operation. 

However, you should always keep in mind that non-functional testing should not be viewed as secondary to functional testing. Both of these testing types have a direct impact on the reception of your software product, so even in situations where the ROI is different from your expectations, it’s not a reason to limit your testing efforts.

Can you skip some types of testing to save money?

There is no standardized set of functional or non-functional tests that need to be performed on the product. In every situation, the scope of testing is defined by the factors we have talked about earlier in this article. However, in most cases, QA engineers run both functional and non-functional tests on a software product. This is done to make the testing process comprehensive and effective, which will result in a thoroughly tested product with a minimum probability of failure.

If we are talking about an abstract software product, then yes, it’s sometimes possible to scale down the scope of testing if you are strapped for resources. However, your testing project still needs to include both functional and non-functional tests. 

The only time you can skip non-functional testing entirely is when you are developing a product that is not intended to be interacted with by regular users, such as a utility software product with no user interface. And even then, you will likely need to test its performance, installation, ability to withstand different loads, and other qualities that factor into the success of a software product.

Why non-functional testing is as important as functional testing when it comes to investing your time and money

The importance of functional testing is usually not a subject of a lot of debate since it deals directly with the intended functionality of the product — the exact things it was designed to do. However, for those who are new to software development and testing, non-functional testing may not seem as essential. Moreover, since non-functional testing tends to be resource-intensive, product owners often wonder whether it’s a worthwhile investment.

The bottom line here is simple: a software product with insufficient or missing non-functional testing is never going to perform as well in the market as a product where every non-functional aspect has been tested down to the smallest possible detail. Although there is more than one important functional and non functional testing difference, these are not mutually exclusive testing efforts. Your job here is not to try to cut down on non-functional testing in an attempt to spend fewer resources, but to find a QA team that is a perfect fit for your needs and budgets.

Final thoughts

Dividing tests into functional and non-functional is only one of the possible approaches to categorizing software testing, but it’s a pretty important distinction. If there is one thing we tried to achieve with this article, it’s to show you that both functional and non-functional testing is crucial when it comes to the success of your software project. 

If you want your product to not only work according to the functional requirements, but also be well-received by the audience, skipping either type of testing or minimizing their importance is a decision that is eventually going to backfire. On the other hand, designing a comprehensive test plan that is tailored both to your project’s specifications and your company’s internal resources, is a strategy that will always pay off.