Wherever you are, you can help Ukraine

Act Now! Flag Ukraine

Regression Testing: Manual or Automated? (with examples)

TestFortExpert by TestFortExpert on 03/4/2014

Regression Testing: Manual or Automated? (with examples)

The question arising at each stage of development is that regression after each iteration takes too long, and the functionality being verified often had no changes. Consequently, the possibility of defects is quite low. As a natural result, we arrive at an idea to automate all regression scenarios and refuse manual regression testing.

The advantages of automated testing here are obvious:

But unfortunately, regression tests automation has many pitfalls that are rarely discussed.

Before you start to automate regression testing is necessary to solve a few questions:

Let us pass on to the real situations in practice. At the stage of switching from manual testing to automated tests, I got a project where I had to test the registration of two types of accounts for the site on 43 domains depending on the country. It was just as hell. The client didn’t care about the quality of the information displayed. The requirements were as follows:

After a couple of runs I realized that I’ve had enough. At that moment I could only see that the number of registration fields is different for some countries, and I also had some coding experience in C#. Another regression was close at hand and I could not hesitate. I asked my friend familiar with automation to tell and show how to write tests. After many questions, tries and errors a simple navigation test was born. I discovered locators and Selenium web drivers – and lo and behold, all the domains used the same locators. I had a little left to do – to run the last manual regression in my life on all domains and fill the table with the fields and corresponding domains. Another round of manual regression – a long process with completing the table – and seems like is all over.

I enjoyed every other regression. I could watch some TV series while regression was running. By the way, it was just as long as one episode. While I used to spend about 4-5 hours on registration testing on 43 domains, automated regression testing took me 40-45 minutes, provided non-optimal code, no report system and no proper exception handlers.

Considering the above said we can make conclusion #1: to see the advantages of test automation, we should start with the most repeated tests, which are usually positive, but they need to be run after each release.

This continued for several releases until the company decided to automate most of the functionality officially. Since the client was not well-versed in automation tests we decided to automate any and all that is within my knowledge on test automation.

As a result, we have covered almost the entire system with autotests. All would be fine, but after any fixes some tests started to fail. Mostly, it happened because of long scripts that contain a lot of steps:

On that basis we can make conclusion #2: too long scenarios should be run manually, as a lot of time is spent on writing them and they can often fail because of some minor bugs or errors at the second step.

Let us consider another very interesting situation. The client ordered autotests as usual and asked to write a couple of examples. He approved the idea, and in several days we added test automation to the plan. But our programmers did not meet the deadline, eventually, we had to write autotests based on the current stage. With a delay of a couple of days programmers had a release, but more than half of autotests failed.

Conclusion #3: never start writing automated test without a stable build.

Only a short time ago we had one more funny situation. We had received an order for site automating and have done everything quickly. Everything went well more than ever. A few weeks later the customer sent a letter complaining that tests did not work. As it turned out, he started a site redesign and all the tests had failed. It taught me to do stable tests.

Therefore, conclusion #4 is: when possible, use the locators so that the elements could be found after the slightest changes.

Therefore, the advantages of manual testing are seen clearly – it doesn’t depend on anything and can be done all the time. Abandoning manual testing will bring nothing good. Automated and manual testing are interrelated and complementary testing methods and each has pros and cos. Before thinking of regression scenarios automation consider time expenditures for both writing tests and their support. Also, pay attention that manual specialists are usually paid less than those who have skills on test automation.

So, as always, the whole issue is about money. If automation of regression testing scenarios saves money without losing the quality of the product, you can expand the range of automated scenarios until the product quality is high and you save costs. The way of balancing here is up to you to decide and depends on personal experience and preferences.

You may also like:

A Quick Guide to Regression Testing

Automated testing with the help of Behavior-driven testing (BDT)

Testing Automation Trends 2018

Top 9 Automated Software Testing Tools

We use cookies to ensure your best experience. By continuing to browse this site, you accept the use of cookies and "third-party" cookies. For more information or to refuse consent to some cookies, please see our Privacy Policy and Cookie Policy