Regression Testing: Manual or Automated? (with examples)

TestFortExpert by TestFortExpert on 03/4/2014

Regression Testing: Manual or Automated? (with examples)

The question arising at each stage of development is that regression after each iteration takes too long, and the functionality being verified often had no changes. Consequently, the possibility of defects is quite low. As a natural result, we arrive at an idea to automate all regression scenarios and refuse manual regression testing.

The advantages of automated testing here are obvious:

But unfortunately, regression tests automation has many pitfalls that are rarely discussed.

Before you start to automate regression testing is necessary to solve a few questions:

Let us pass on to the real situations in practice. At the stage of switching from manual testing to automated tests, I got a project where I had to test the registration of two types of accounts for the site on 43 domains depending on the country. It was just as hell. The client didn’t care about the quality of the information displayed. The requirements were as follows:

After a couple of runs I realized that I’ve had enough. At that moment I could only see that the number of registration fields is different for some countries, and I also had some coding experience in C#. Another regression was close at hand and I could not hesitate. I asked my friend familiar with automation to tell and show how to write tests. After many questions, tries and errors a simple navigation test was born. I discovered locators and Selenium web drivers – and lo and behold, all the domains used the same locators. I had a little left to do – to run the last manual regression in my life on all domains and fill the table with the fields and corresponding domains. Another round of manual regression – a long process with completing the table – and seems like is all over.

I enjoyed every other regression. I could watch some TV series while regression was running. By the way, it was just as long as one episode. While I used to spend about 4-5 hours on registration testing on 43 domains, automated regression testing took me 40-45 minutes, provided non-optimal code, no report system and no proper exception handlers.

Considering the above said we can make conclusion #1: to see the advantages of test automation, we should start with the most repeated tests, which are usually positive, but they need to be run after each release.

This continued for several releases until the company decided to automate most of the functionality officially. Since the client was not well-versed in automation tests we decided to automate any and all that is within my knowledge on test automation.

As a result, we have covered almost the entire system with autotests. All would be fine, but after any fixes some tests started to fail. Mostly, it happened because of long scripts that contain a lot of steps:

On that basis we can make conclusion #2: too long scenarios should be run manually, as a lot of time is spent on writing them and they can often fail because of some minor bugs or errors at the second step.

Let us consider another very interesting situation. The client ordered autotests as usual and asked to write a couple of examples. He approved the idea, and in several days we added test automation to the plan. But our programmers did not meet the deadline, eventually, we had to write autotests based on the current stage. With a delay of a couple of days programmers had a release, but more than half of autotests failed.

Conclusion #3: never start writing automated test without a stable build.

Only a short time ago we had one more funny situation. We had received an order for site automating and have done everything quickly. Everything went well more than ever. A few weeks later the customer sent a letter complaining that tests did not work. As it turned out, he started a site redesign and all the tests had failed. It taught me to do stable tests.

Therefore, conclusion #4 is: when possible, use the locators so that the elements could be found after the slightest changes.

Therefore, the advantages of manual testing are seen clearly – it doesn’t depend on anything and can be done all the time. Abandoning manual testing will bring nothing good. Automated and manual testing are interrelated and complementary testing methods and each has pros and cos. Before thinking of regression scenarios automation consider time expenditures for both writing tests and their support. Also, pay attention that manual specialists are usually paid less than those who have skills on test automation.

So, as always, the whole issue is about money. If automation of regression testing scenarios saves money without losing the quality of the product, you can expand the range of automated scenarios until the product quality is high and you save costs. The way of balancing here is up to you to decide and depends on personal experience and preferences.

You may also like:

A Quick Guide to Regression Testing

Automated testing with the help of Behavior-driven testing (BDT)

Testing Automation Trends 2018

Top 9 Automated Software Testing Tools

We Work With

Having one outside team deal with every aspect of quality assurance on your software project saves you time and money on creating an in-house QA department. We have dedicated testing engineers with years of experience, and here is what they can help you with.

Software is everywhere around us, and it’s essential for your testing team to be familiar with all the various types and platforms software can come with. In 21+ years, our QA team has tested every type of software there is, and here are some of their specialties.

There are dozens of different types of testing, but it takes a team of experts to know which ones are relevant to your software project and how to include them in the testing strategy the right way. These are just some of the testing types our QA engineers excel in.

The success of a software project depends, among other things, on whether it’s the right fit for the industry it’s in. And that is true not just for the development stage, but also for QA. Different industry have different software requirements, and our team knows all about them.

Icon Manual Testing

Maximum precision and attention to detail for a spotless result.

Icon Testing Automation

We’ll automate thousands of tests for all-encompassing coverage.

Icon Testing Outsourcing

Outsource your testing needs to a team of experts with relevant skills.

Icon Testing Consulting

Overhaul your QA processes to achieve even more testing efficiency.

Icon QA

Thorough Quality Assurance for a project of any scale or complexity.

Icon API Testing

Verify the correct operation of as many APIs as your project needs.

Icon IoT Testing

Stay ahead of the growing Internet of Things market with timely testing.

Icon Web App Testing

Reach out to even more customers with a high-quality web application.

Icon Mobile App Testing

Help users fall in love with your mobile app with our texting expertise.

Icon CRM/ERP

Make sure your CRM/ERP system meets the needs of the stakeholders.

Icon Desktop Application Testing

We’ll check the stability, compatibility, and more of your desktop solution.

Icon Functional Testing

Is your app doing everything it’s supposed to? We’ll help you find out!

Icon Compatibility

Check how your solution works on different devices, platforms, and more.

Icon Usability

Find out if your software solution provides an engaging user experience.

Icon UI

Make sure your application’s UI logic works for all categories of users.

Icon Regression

We’ll verify the integrity of your application after recent code changes.

Icon Online Streaming & Entertainment

Stay on top of the media industry with a technically flawless solution.

Icon eCommerce & Retail

Does your store meet customer needs? We’ll help you know for sure!

Icon HR & Recruiting

Streamline HR processes with a solution that works like a clock

Icon Healthcare

Test the functionality, stability, scalability of your app and more.

Icon Fintech & Banking

Give your users what they want: a powerful, secure fintech product.


We use cookies to ensure your best experience. By continuing to browse this site, you accept the use of cookies and "third-party" cookies. For more information or to refuse consent to some cookies, please see our Privacy Policy and Cookie Policy