What is smoke testing?
Smoke testing is a type of software testing that determines whether a new build of the software product is stable and ready for further testing. In other words, it tells the QA engineers whether the build has the required functionality or it needs to be sent back to the development team. Smoke testing is often referred to as “build verification testing”, and this term is pretty self-explanatory.
The actual term “smoke testing” was coined by engineers performing electronic hardware testing. When a new board is plugged in and there is suddenly smoke, it means no further testing can be done at this moment — all that is left to do is to turn off the power and go back to the previous stage.
Why is smoke testing important?
To those who don’t work with software testing on a regular basis, smoke testing seems like an additional step in the testing process. However, there are three major reasons why smoke testing should not be overlooked:
- Smoke testing is the first indicator of software quality. When a build fails smoke testing, it means that the product has significant problems with quality which, when left unchecked, can lead to bigger issues, including loss of profit.
- Smoke testing helps save time and money on testing. Smoke testing a software build is often the first round of testing in many QA teams. The earlier a bug is discovered, the less time it will take to fix it and the cheaper it’s going to be.
- Smoke testing helps identify critical issues. The nature of smoke testing deals with the most significant bugs, not minor issues with the UX or compatibility. These are critical issues that often prevent the software product from starting or running smoothly at all.
When is smoke testing performed?
By now, you already know that smoke testing happens early in the software development and testing life cycle. But when exactly should it take place and how many times should it be performed?
Generally speaking, smoke testing is performed every time there is a new software build intended to be released. This includes both brand new software solutions and builds that are created to add new functionality or improve the existing features. This is why smoke testing almost never happens just once: like regression testing and re-testing, it’s part of the effort to make sure that the changes in the software did not have an undesired effect on the quality.
Smoke testing vs. sanity testing vs. regression testing
Smoke testing, sanity testing, and regression testing happen nearly at the same time in the software development life cycle. This is why it’s easy to get the impression that these types of testing are interchangeable, and if you perform one, you can skip the others. Others think of regression testing as being the most important one, while the other two can be skipped without consequences.
However, while smoke testing, sanity testing, and regression testing have similarities, they are fundamentally different types of testing that have different purposes. Here is why they should not be confused and why they all matter:
- Smoke testing is used to assess the stability of the build overall before other types of testing are carried out. Smoke testing can be done both manually and automatically and is typically one of the first types of testing performed on a new build.
- Sanity testing exists to test the status of the system that is about to be tested. It can take place immediately after a successfully passed smoke testing phase and is usually done manually, although automated sanity testing techniques exist as well.
- Regression testing serves to make sure that the recent changes to the code of a software product did not negatively affect the existing functionality. Regression testing can be done manually, although automated regression testing makes the process more efficient.
How to do smoke testing in software testing?
We’ve already mentioned that smoke testing can be done both manually and automatically. However, when a QA team is getting ready to run smoke tests for the first time, they still need to prepare the test cases and create a test environment from scratch. Here are the steps to take to perform smoke testing the right way:
- Preparing smoke test cases. This step can be done together with the development team, who can help identify the most relevant cases and create scenarios that will thoroughly test the product without wasting time on non-essential tests.
- Ensuring a clean testing environment. In order to yield unambiguous results, the testing environment needs to be clean and free of anything that can hinder the testing process or manipulate the outcome of smoke testing.
- Running the tests. When both the environment and the test cases are ready, a QA engineer can run the tests according to the prepared test scripts. At this stage, the QA team can already note potential defects and relay the information to the developers.
- Analyzing the results. After all tests in the test plan are completed, it’s time to analyze the results. Specifically, the team determines whether the current software build is successful or not. A successful build is then sent to the functional testing stage.
- Creating documentation. Smoke testing is not typically designed as a one-time activity. Every time there is a new build of the software product, smoke testing is required. This is why detailed testing documentation is essential: it saves the time it takes for the QA team to execute the tests for every new iteration of the software.
Compared to functional and non-functional testing, which include the most essential aspects of a strong software product, smoke testing can seem not as important. However, most QA teams agree that smoke testing should not be viewed as an optional stage. Timely smoke testing can save the team a lot of resources that can be used to further improve the product.
