Configuration testing plays a critical part in the development life cycle by specifying the quality and portability of software. It helps to understand whether a program can be used on any operating system other than the one for which it was designed. Or whether there is any work required to make software run in the new environment.
Let’s find out what is configuration testing with an example, and how does it work?
What Is Meant by Configuration Testing?
Configuration testing is a type of testing that is performed to verify the best possible performance of a system and the least and most appropriate configuration that does not result in bugs and defects.
Configuration testing ensures your app functions with as many different hardware elements as possible. For this purpose, it is tried out on different supported system configurations, by which we mean combinations of operating systems, browsers, drivers, etc. For example, Oracle Database and MySQL (databases), Chrome and Microsoft Edge (browsers), etc. The idea is to figure out whether the app is suitable for all of them or not. Every combination of software and hardware is incorporated into the testing process. The team detects the most suitable one among them.
And now let us consider what is meant by configuration testing from the perspective of its objectives.
Objectives of Configuration Testing
Even though the hardware is expected to be designed to a set of standards to function without delays and failures, there are cases when it does not work properly with specific hardware configurations. This happens because not all of the standards are kept.
The primary purpose is to distinguish the optimal configuration of equipment that outputs the expected behavior attributes of a system and check its compatibility with the declared hardware and operating systems. It assists testers in system performance and availability validation when there are modifications in configurations.
The basic objectives of configuration testing are:
- Partial app validation;
- Requirement failure identification (to check whether an app satisfies configurability requirements);
- Failure identification and minimization (to discover bugs that were missed out or not discovered productively during different testing phases);
- Performance verification regardless of system settings;
- Hardware modification evaluation;
- Identification of the most appropriate system configuration;
- App verification (it relates to how manageable app elements are during the entire development cycle);
- Software app analysis (by switching software and hardware resources).
How to perform configuration testing?
The process has a set of prerequisites or activities that should be kept to in the beginning. Here is how to do configuration testing correctly.
- A team creates a covering matrix with all possible variations of software and hardware setups. It is hard to check each and every variation effectively when there are so many potential ones available (e.g., 800 or more). Thus, it is crucial to underline the exact platforms that are going to be supported.
- Configurations are placed in order of importance. The most important ones are tested in detail first.
- Each of the previously prioritized combinations is checked by how it has been determined.
The phase of planning of configuration testing should never be skipped.
A team decides on the hardware pieces, features, modes, and options that are needed for the software to work. Not all of the hardware features or modes of a device should be supported by the software. Thus, only the necessary ones should be picked out. What is the set of requirements for features and options for a configuration to include to run them? A set of unique features of the software that run with the hardware combinations are determined. There is a limit to the number of probable scenarios that can be used by a tester. Thus, only those that matter are left.
A team can create a table to compare potential combinations by putting together any related information in one place. What are the unique software features that support those hardware patterns? The test cases are created for every configuration. If a bug occurs, it is important to distinguish whether the reason is hardware or software. The results are reported to a team or a hardware manufacturer.
Remember, preparation for the configuration test is the little bits and pieces to guide a tester through the process.
Types of Testing
There are two types of testing when it comes to system behavior analysis with multiple configurations.
Software Configuration Testing
It comes after unit and integration testing and involves numerous operating systems, browsers, and platforms to compare and check whether an app is compatible with every potential piece of software or not. The testing environment is reset over and over again, and the app is validated by running on different software. The procedure takes some time to complete as both processes of installation and uninstallation are laborious. To decrease time spent on software test scenarios, a team can use virtual machines. They mimic real-time configurations, saving costs and increasing productivity.
Hardware Configuration Testing
This one comes after a new build is presented and assumes components, which are connected to physical machines, to check hardware combinations and the expected functioning of an app. The main task is to have various machines, on which you can set up the software when a new build appears. Usually, a team automates the process for a specific set of hardware combinations because it requires substantial time and effort. However, this does not make the process simple instantly. It is challenging to complete 100% of all the tests. Thus, configurations are analyzed and the most used hardware is selected and prioritized to conduct testing. The most likely configurations are selected among thousands of alternatives.
There are two more types of configuration testing, divided on the basis of testing level.
- client-level;
- server-level.
A client-level technique has a client’s perspective and a similar concept as usability and functionality testing. A server-level technique is focused on the integration after a release and analyzes the communication between software and the external environment.
Conclusion
If configuration testing is not implemented, the risks of the configured elements not being attached to the software and the system collapsing are increased. Previously developed measurements for handling various configurations matter. They make it possible for an app to interact with hardware and fulfill configurable requirements as expected, contributing to a bug-free application.
