There are still many business owners, especially the ones fresh in the industry that believe test automation is like a magic wand that speeds up tests, reduces costs, increases coverage and delivers coffee to bed sort of thing. They don’t really calculate all the details. Test automation is a great tool for achieving precise goals, not more nor less.
Most test automation benefits are long-term and might not even be visible from the start. However you still do require making sure there are any benefits at all and that is when you will require to calculate ROI aka return on investments. So, simply spoken, ROI is of your automated tests should be measured as the division of tests automation benefits on their costs.
How to Calculate?
You have to realize that automation is clearly more expensive than manual testing and also involves hardware and software expanses as well as additional costs for licenses. You will also require resources for script production that are also worth money. These are the expanses in general, but how can one really measure benefits?
This part is a little trickier but not that hard. You take a certain period of application testing progress and measure the time reduced with automation combined with test frequency. Do the same for manual tests of the same application and compare results.
Here is a little something for even better understanding:
- automation costs = $ for required hardware + $ for appropriate tools + time-&-resources required for script development + script maintenance and execution time for the first time costs + the same for multiple executions
- manual costs: time-&-resources required for script/user story development + script maintenance and execution time for the first time costs + the same for multiple executions
- ROI = manual testing costs – automation testing costs
Is it really as simple?
Well, no, the listed above is just the most applicable concept and every single project will have a different ROI due various tasks and availability of resources. Sometimes automation may not even be the best choice. Make sure you have place for manual tests in either case especially in such vital spots as User Interface or Usability. There are also projects when the software under test is not too large and manual tests will not be taking long and such cases would be a simple waste of resources if you would turn to automation. However the general concept described above should make the choice on whether or not you need automation easier and more efficient.