How to establish a successful testing process for your product

Testing has always been a critical factor for delivering bug-free, reliable and secure products. With the high competition on the market and increasing customer demands the benefits testing can bring you are huge. Let’s say you have already found the right testing team. This is only a part of the solution. Now you need to organize a testing process the right way.

As a client, you may think that the testing provider defines everything about your testing project. And to some extent you are right, but that is just a part of the bigger picture. Testing is a precise process and each stage of it is extremely important as it can affect the results. The first part of the testing project — the part before the testing actually starts — is not an exception. The smooth start of each testing project very much depends on the communication of the testing team and the client. To be more exact, the more information you provide, the greater the chances you will successfully carry out the tests, which will lead to a better performing product. Here are some questions to keep in mind while planning a testing project.

QTNA: The basic list

Although much depends on the type, scale and specific characteristics of the project, typical questions from the testing team include the following:

1. What are your overall expectations from testing?
2. Was the testing process performed on this project before?
3. What are your expectations for the product?
4. What environments are needed for testing?
5. What tools and infrastructure should be used in the project?
6. What test artifacts will be used during the testing process?
7. What are the timing of the testing process and the frequency of product releases?
8. Does your team have a testing strategy?
9. How should the testing process be reported?
10. How should communication between the customer and testing team look like?

Let’s explore these questions in greater depth.

The expectation from testing

The first question to be answered is the question about clients’ general expectations from the testing activities. In particular, you can be asked about the circumstances and reasons which prompted you to test your product in the first place. It could be a lot of things, including the necessity to improve the quality, the reduction of the development team or simply a desire to learn more about the product (UI/UX) for its further improvements. Once the testing team knows the goals of the testing project the options of its implementation can be considered.

Was testing performed before?

The other thing that the testing team needs to know is if the testing took place before. This provides the team with understanding whether you know what testing is all about, or they should establish the testing process from scratch. If the product has been tested before the client is usually able to share some information about the process and the results achieved. This, in turn, will help the testing team to figure out which approaches are worth offering and which are not.

What do you expect from the product?

Once the team knows the purpose of the testing project the next thing they will want to establish is your expectations for the product. The first important question is “Who is your target audience? In other words, you should tell the testing team about the people or the market for which your product is intended. You should also settle on the priorities of the product. What should testers be focused on — particular functions, performance, security, UI or scalability of the product? The answer to these questions will have an impact on the testing process, in particular, on the types of tests they use. 

What about the testing environment?

It is equally important to outline the environments on which the product will be tested. Nо wonder that you want your solution to work on all platforms, browsers, and devices to reach the biggest audience possible. But in reality you will need too many resources to cover all the environments. At the same time, the right priorities will allow to reduce the testing time, improve the quality of the test coverage and detect more bugs. Choosing the right number of environments for testing has never been easy. Luckily, there are good tools that might help. Good examples include Google Analytics and Yandex. Metrics that show on which platforms, browsers, and devices, the product is most often tested.

The question on environments is critical as testers need to know whether they have all the necessary platforms and devices to test the product. If they do, everything is fine, but if they don’t, they will have to make further efforts to configure virtual machines for the missing platforms and/or use emulators/simulators for the missing devices. It is important to discuss all the requirements to the testing environment and prepare everything in time.

Testing tools and infrastructure

You should also think in advance about the tools and infrastructure for the testing process. The testing tools may include:

• Bug tracking systems (Jira, Redmine, Mantis, Trello, etc.);
• Systems for creating test documentation, as well as for coordinating the testing process (TestRail, TestLink, etc.);
• Supporting tools (for screenshots, removing logs, working with the database, SOUP UI for XML, etc.);
• Automation tools (TestComplete, Selenium, etc.).

By infrastructure, we mean the testing environments, which, depending on the product scale and goals, may include:

• Development environment;
• System testing environment;
• Integration environment;
• Stress testing environment;
• Prod-like/Pre-prod environment.

By choosing the testing tool you have to remember:

• What tasks should be accomplished?
• What budget for the testing tools do we have?

The answers to these questions will allow to determine the best testing tools for your project. You should also agree with the testing team what test environments they should use and provide access to them, so they have enough time to set everything up.

Test artifacts

Although all experienced testers know what test artifacts should be used, additional clarification never hurts. Are there any test artifacts you already use? Do you have a test plan? Are there any checklists (which ones)? Are there any written cases? Which bug report templates do you use? The answers will help the testing team to understand what test artifacts they have to create, calculate how much time it takes, and agree on the formats of the test artifacts for the testing project.

More questions about the product

The testing team can always ask some additional organizational questions about the product. For example: when is the scheduled release date, or how often your product will be updated, or which methodology do you use to manage your product? The information you provide to your testing team will help them plan the testing process better by spreading the testing activities in accordance with the set deadlines.

Testing strategy

The next step is to discuss the testing strategy with your team. First, you should agree on the most critical functionality of the product, to draw their attention to it. Then the team will choose the testing method and show you what exactly will be tested and how much time it will take. The professional testing team will always explain to you the reasons for their choice and offer you several alternative options. With the right testers, you will be also informed about the consequences of waiving particular types of tests or from speeding up the testing process. This approach is the only way to create a testing strategy that will suit your business needs best.

Project reporting

Testing helps collect various qualitative and quantitative indicators for your product (the same goes for the project) and maintain statistics. In order to keep it organized, you and your testing team should agree on the format of the test results. This could be a traceability matrix, graphs, or charts — anything you need. The most important thing is that this information must be visible, accessible, and understandable. As a customer, you can also request to send you daily/weekly activity reports.

Communication

Communication is a king, they say. You should be very clear about how you see the communication process with your testing team from the beginning. How often do you want to receive the updates from them? Do you practice everyday meetings on your project? Are you planning to hold any training meetings for the testing team? Do you need retrospective meetings? Those are the questions that need to be answered too.

Final thoughts

The answers to the questions described above will help you to overcome/prevent the obstacles during the testing process and contribute to the general project success. Except for catching the bugs, testing also provides you with a great amount of useful information that may help to improve the quality of the product in the future. The more well-thought-through your testing process is, the greater your chances to launch a high-quality product that works as it should and bring the best experience to the users.