Building an Effective Test Automation Team: From Hiring to Management

By some estimates, up to 100% of software companies use automation testing in some capacity. There’s a lot been said about the importance and benefits of automation, including on our own blog. Plus, we’ve already talked about building an efficient test automation strategy and choosing the right set of automation testing tools. Today we are going to talk about another core aspect that is at the heart of any test automation project: an automation testing team.

What does an automation testing team do, how to build one with the best available automation testers, and how to ensure its effectiveness throughout the entire automation project? We’ve got all the answers you need right here.

Who should do automation testing: a manual QA team, a development team, or a separate team?

Test automation is a unique subset of software engineering: it’s a piece of code that is created with the sole purpose of testing another piece of code. This is why many software companies feel like it’s only natural to entrust the process of automation testing to the developers. Other organizations go in a different direction by creating an automation team from scratch. And some companies choose to grow their own automation testing departments by turning manual testing QAs into test automation engineers. So which strategy is the right one?

The answer to this question depends on the company’s available resources and both short-term and long-term goals. For example, when the company is planning a long-term automation project that is expected to grow both in size and complexity, and there is already a skilled and motivated manual QA workforce, then helping them switch to automation seems like a reasonable move.

The downside of this solution is that it takes time to acquire at least some knowledge of automation, and in the end, you’ll get Junior or Strong Junior automation engineers at best while still having to employ the services of a Senior Automation QA or a Lead. However, those can be arranged on an outsourced or freelance basis, so it shouldn’t become an issue.

In the case of software developers turning into automation engineers, those situations are fairly rare and may not yield the best results for one reason: after spending so much time honing their coding skills, software developers may unconsciously try and write code that is guaranteed to pass the tests, not discover as many defects as possible. Therefore, the results of testing can be biased.

All things considered, building a test automation team from scratch is probably the best option, provided that you have the resources and are confident that the scope of the project will keep your new automation QA department occupied long-term. You can hire automation testers in-house, choose to outsource instead, or use a combination of the two methods — more on those later in our article.

Typical roles and responsibilities in an automation QA team

The automated test team structure in software testing is not that different from the setup of a manual team, although the exact composition and responsibilities of an automation software testing team structure will still be different from manual QA. These are the must-have automation testing roles and responsibilities:

• Automation QA Lead. This is the person responsible for setting up the automation project and overseeing it throughout all the phases. An Automation Lead mostly deals with the big picture, such as identifying the scope of testing, selecting the framework and the tools, and ensuring smooth communication between every party, including the project stakeholders. It’s worth noting that the work of Automation QA Leads goes way beyond writing the code, even though they are definitely capable of it.
• Senior Automation QA. When the project is big enough, there is certainly room for both a Senior and a Lead Automation QA. However, sometimes organizations choose to have just one of the two. A Senior Automation Engineer also deals with the big picture, but a lot of their work also concerns the day-to-day execution of the project strategy, as well as helping junior members of the team get a full understanding of the task in front of them and the tools they need to use. Typically, senior engineers will also write code for tests.
• Infrastructure Engineer. This is a dedicated person for handling the technical details concerning the launch and progress of an automation project, such as the health of an automation test suite and the framework, as well as remote test execution. This position is usually reserved for large-scale and long-term projects, as there may simply not be enough for a Testing Infrastructure Engineer to do on a smaller project. On a mid-sized project, this role can go to Lead or Senior QAs, and on larger projects, the DevOps department can handle these tasks.
• Junior and Middle Automation QAs. This is the biggest part of the workforce of a typical automation testing project and the one role every test automation team must include. Junior and Middle dedicated automation testers are involved in every stage of the project, from developing the test suite and setting up the automation tools to generating and analyzing the reports, using the data to make the testing process more efficient and resource-effective.

Other project team members may include a project manager, a product owner, and a software developer for handling code-related issues. Moreover, even though the goal of test automation is to lower the dependency on manual testing, we still cannot completely remove manual testers from the equation, as there are still areas that automation cannot fully cover. It’s safe to say that successful test automation requires at least some input from manual QAs.

Automation roles and team structures: Which one to go for?

Whether you decide to hire dedicated automation testers for a one-off project or strengthen your in-house QA team to focus on long-term goals, one of the most important choices you’ll have to make is who will lead the entire team. Automation is a team effort, but there is someone who needs to take charge and set direction. These are the main team types to consider.

Developer-led team

When the dev team or product team is the one team performing the complete scope of development and testing tasks, the developer with the most experience in development and at least some understanding of manual and automation testing can assume the role of an automation team leader. This brings some tangible benefits to the process, including:

• Test automation code is written alongside the product code.
• Failed tests can be corrected quickly.
• Members of the team manage and prioritize their own scope of tasks.
• Fewer outside resources are spent on automation testing efforts.

At the same time, choosing a developer-led team composition has some disadvantages as well:

• A developer rarely possesses the automation skills and knowledge of test automation solutions that a software test automation engineer is required to have.
• Automated test cases need to be regularly maintained and reviewed, so if the team doesn’t have enough resources for that, test automation scripts can become neglected and obsolete.
• Establishing and maintaining the test automation procedure typically requires the leader to take time off their main duties.

Tester-led team

A tester-led team dealing with test automation is a team that has almost complete autonomy and can manage both manual testing and automated testing without much interference from the development or product teams. This type of team is usually a distinct group of automation staff with both automation and manual backgrounds plus a strong leader who is responsible for creating a testing strategy, overseeing the writing of test scripts, and providing guidance that the team needs. Tester-lead teams have a few strong advantages:

• QA engineers can choose or develop a strong test automation framework, select the most appropriate test automation tool set, and pick realistic goals and milestones.
• There is clear accountability, as you always know who is responsible for a particular task and knows all the details of the progress and the outcome.
• When one team does all the manual and automated testing, there is a continuous knowledge-sharing process that produces high-class specialists without the need to grow the team.

Still, there are some disadvantages to this setup as well:

• The quality of the test automation suite and the code in particular largely depends on the skills and experience of the members of your team.
• An automation team can support only so much of the testing process without expanding or further training, meaning either one of them or both will be required in the future.

Split effort-led team

This is a setup that is best suited for Scrum teams, where both developers and testers are involved in the testing process. Automation testing can work well and deliver results quickly when the developers write automation scripts for new features they’re developing and then hand them over to the testers for execution, maintenance, and updates. This seems like the perfect team composition for a number of reasons:

• Effective test automation is achieved through coding best practices.
• Shared responsibility for the project outcomes means every team member can see their input and not be burdened by the enormous scope of tasks.
• Development and testing happen simultaneously, facilitating bug detection and resolution in the early stages of the product life cycle.

While the advantages of this setup are hard to argue with, there are some drawbacks to consider as well:

• In addition to their day-to-day tasks, developers also have to deal with automated testing challenges.
• Automation testers need to have significant skills in the field to not just create their own tests, but also to manage and maintain tests created by the developers.
• As the project progresses, the team will inevitably have to make some compromises in test coverage, and it can be difficult to avoid scope creep without further expanding the team.

What’s the right size for an automation team?

When discussing how to build a software testing team that will deal with automation, the topic of team size is one of the most controversial yet most important ones. The reason why organizations care so much about the ideal team size is completely understandable: on the one hand, they don’t want an overblown staff; on the other hand, they don’t want to stretch their available resources too thin.

Unfortunately, there aren’t any formulas with a 100% success record that can help find out the ideal size or composition of a good automation testing team — ultimately, it depends on the specifics of the project. However, many automation experts agree that having one to two automation engineers per four or five software developers is an approach that usually works.

Of course, it’s also important to take into account the seniority level of the automation QA: the higher the level, the more challenging tasks they will be able to complete in the same amount of time. However, “one automation QA to four or five developers” is a formula that can help you get started on a brand new project and then adjust your resource allocation as you go.

In-house or outsourced: Which one is right for your test automation strategy?

As it’s often the case with outsourcing your software needs or choosing to do everything in-house, this question is deeper than it may initially seem and requires thorough consideration before any decisions can be made. Should you hire test automation engineers in-house, or is it better to outsource your test automation efforts? Here are a few reasons why outsourcing all or some of the test automation tasks can benefit your project:

• Niche expertise. When you are no longer limited to your local area in your hiring decisions, you get to work with engineers with a specific set of skills or one-of-a-kind expertise that can be rare to find in the industry.
• Financial benefits. Compared to the US and other Western countries, the work of an outsourced automation QA in Eastern Europe or Asia can cost you several times less for the same selection of services. Moreover, you don’t have to worry about office space, insurance, and other overhead costs.
• Flexibility. Working with an outsourced automation QA engineer or a whole team always happens on your terms, as you get to determine the scope and duration of the project, team size, and so on. You can even create a unique automation setup at your organization — for example, have the in-house team work for you full-time and outsource the Automation QA Lead position to an offshore engineer for 10 hours a week — which is rarely possible with a fully in-house project.

One of the concerns keeping some organizations from fully taking advantage of the outsourcing model is that managing and evaluating a remote team is not as easy as when everyone is working from the same premises, as test automation requires continuous cooperation between the engineers to be successful. However, if the past four years have taught us anything in terms of job models, it’s that remote and hybrid work are only going to become more common and that different locations and time zones do not stand in the way of efficient management.

How to build an automation QA team that meets your needs

When building an automation software testing team from scratch or strengthening your existing test automation department with new team members, selecting and hiring the right people is a task of utmost importance. Here is how to develop high-performing teams for automation.

1. Create a solid vacancy description

The key here is to make the description as relevant to the job as possible, and not just to copy and paste the requirements from another vacancy. A difference in the requirements and duties between the job description and the actual interview will create a lot of confusion and damage the candidate’s opinion of the company. This is why it’s always a good idea to collaborate with someone who is actually working on the same project to outline the description, as well as ensure smooth two-way communication between the departments in case the project is to be launched from scratch.

2. Allocate enough time for hiring

Assembling an automation team from scratch or getting one or two strong additions to the team is not something that happens overnight. Things can get even more challenging when you are looking for senior-level engineers or automation leads, as they are no longer motivated to frequently switch jobs and are typically pretty happy where they currently are. The two things more likely to make them change their mind are an interesting tech stack and higher compensation.

The rule of thumb is that the more seniority you want to see in the position, the more time it will take to sign the contract. On average, it takes around a month to hire one senior automation engineer. It’s also worth noting that a currently employed candidate may need to fulfill their existing contract requirements, which can take another two to four weeks.

3. Know what to look for in an automation engineer

A competent automation QA combines both hard and soft skills that make them good at their job, and you can usually tell about some of those skills being present or missing already at the screening stage. Here are some of the things to pay attention to in an automation engineer’s CV:

• How many years of experience they have in total, how many years they’ve spent on average in every organization, and whether their experience in the last workplace is a logical continuation of their overall experience;
• How detailed the description of the last projects is and whether the type of product, platform variety, tech stack, and toolset match the requirements of the project;
• Whether there are any inexplicable gaps in the professional experience;
• The overall look and feel of the CV: it doesn’t have to be overly polished or overly creative, but it does need to be well-written, informative, and free of mistakes purely out of respect for anyone reading it;
• Whether the candidate goes into detail when explaining their role, responsibilities, and accomplishments on the project, or simply lists the basic project specifications that may not even fully match their seniority level.

If you want to get a better idea of the candidate’s personality, you can also check the section where they list their qualities and hobbies outside of work. However, it’s only natural for people to try and present themselves in a more positive light whenever possible, so take those words with a grain of salt.

4. Complete the interview stage

Ideally, the interview — especially the technical one in case the company uses multiple interviews to assess one candidate — should once again be conducted by a person with immediate knowledge about the process and the tech stack of the upcoming or current project. That way, the company will be able to choose a candidate with the most relevant skills and experience. But since an interview is a two-way process, it’s also important to give the candidate a realistic picture of what the project entails and what is expected of them.

5. Assign the leadership role

A successful test automation team requires a clearly identified leader, whether it has two, five, or twenty members. And it’s not always the person with the most seniority or the person with the longest CV. It should be the person who is experienced in building test automation from scratch, maintaining it, and training other members of the team. The lead should also have the ability to write test scripts and possess a lot of proven technical skills. Moreover, this should be a person with a realistic idea about automation in general, thinking of it as a helpful tool rather than the ultimate answer to every quality-related question or a complete replacement for manual QA in general.

6. Consider bringing additional specialists on board

Many organizations want to go the lean route when establishing an automation QA department, but the truth is that it’s nearly impossible to build high-performing teams for automation without eventually adding niche specialists on board.

This can be a principal developer with a strong background in methodology, a project manager with experience encompassing software development and quality assurance, an infrastructure engineer, or a person who will deal with setting up the reports and processing them for maximum efficiency.

7. Lead by example

Whether every member of your automation team is well-versed in their craft or there are some junior members who believe automation is challenging and therefore need time to master automation as a whole and the project specifications in particular, the best way to launch an effective automation QA operation is to go all-in.

Identify the scope of tasks for each member, assign the people responsible for different aspects of the project, tell your junior team members who they can turn to in case they have questions, and launch the project, showing how it should be done.

Automation team culture: How to create a good one and why it matters

Team culture is not always the number one idea in the mind of a person running an automation department, and it’s completely understandable: with so many technical matters to take into account and so many goals to meet, culture risks becoming an afterthought. So does team culture matter for automation QA, what does it entail, and how to maintain it at the right level?

The important thing to note here is that within an automation testing project, team culture is more about sharing the same work ethic and approach to solving problems than about sharing the same interests outside of work or having a similar sense of humor, although those definitely help in establishing a good rapport within the team.

When you build high-performing teams for test automation, team culture can be fostered in the following ways:

• Knowing the end goals of the automation project and which results will matter the most;
• Fully believing in the importance of automation as the key way to achieve those goals;
• Demonstrating firm leadership but also knowing when and how to delegate tasks;
• Promoting the idea of constant learning and mentorship among the team members;
• Creating an environment where people can communicate and exchange ideas openly;
• Gradually lowering the amount of control as the project progresses successfully;
• Locating the ideal balance between speed and quality without compromising one or the other;
• Adopting the practice of stopping for a moment to get a bird’s eye view of the project;
• Making an effort to get to know each other not just from the professional, but also from the personal standpoint;
• Encouraging the team to spend time outside of work, even in a virtual setting, but not insisting on it;
• Establishing effective communication across teams — most importantly, with the software development and the manual QA departments;
• Avoiding the desire to make the team completely homogenous, as variety and thinking outside the box fuel innovation and drive progress.

How to manage an automation team for maximum efficiency

“How to develop high-performing teams” is one of the key interests for thousands of organizations globally, including ones that deal with automation. When you assemble a dream team of automation QA engineers, the last thing you want is to have them not live up to their full potential due to poor management or wrongly distributed tasks. Here are a few tips on how to make the most of your automation software testing team.

1. Choose roles and set responsibilities correctly

There are few things more discouraging to a team member than having to deal with a task that doesn’t match their area of expertise or seniority level, and this can go either way: a Junior team member may be overwhelmed by an overly complex task, while a Senior team member may experience burnout from having to repeatedly deal with tasks below their skill level.

This is why any automation QA project needs to have an experienced Lead in the role of a manager. The Automation Lead will be able to correctly identify the skill level and area of expertise of each team member and assign tasks accordingly.

2. Allocate time for training and team building

Whether you are starting an automation project with a team assembled specifically for this task, a team of manual QAs ready to take on some new responsibilities, or a team of software developers with an interest in automation testing, you cannot expect every single team member to just jump head first into a completely new work environment and thrive there.

Reasonable training time should be mandatory for any new automation-related project. This can be the time for the team members to familiarize themselves with the scope of the project, the objectives, the framework and the tools, and, more importantly, with each other. These are going to be the people working on the same project day in and day out, so make sure they know each other on a decent level.

3. Keep your team informed throughout the project

We’ve already talked about the importance of good communication within an automation team, and that involves communication not just between the team members themselves, but also with the management.

The situation where each team member is constantly in contact with all of the stakeholders of the project may definitely be overkill. However, keeping your team in the loop at every stage of the project is integral for their complete understanding of the project specifics and their work towards the end result. This also includes investing effort into communicating the goals of the project to every team member in the very beginning, so that everyone involved knows what they are doing it for.

4. Don’t get too focused on metrics

There are different ways to measure the effectiveness of a test automation team, and various metrics are among the most commonly used ones. The exact set of metrics can largely depend on the project’s characteristics, although some, including the number of test cases automated, total test duration, number of user stories covered in a week/month, number of defects located through automation, are applicable to nearly every QA project.

At the same time, there is a risk of becoming too dependent on metrics. When the team’s leadership is all about the metrics and the team is aware of that, there is a real risk of the team treating metrics as the only way to know if they’re good at their job. Instead, the team members and the management should work together towards a bigger goal, such as the quality of the product overall.

5. Promote out-of-the-box thinking

The work of an individual automation QA engineer or the whole team should not be just a sequence of tasks needed to get from point A to point B with a predetermined selection of tools and solutions. While following standard procedures is definitely beneficial to a project to some degree, creativity and out-of-the-box thinking are at the heart of being a successful automation QA.

This is why the person managing an automation testing team should always encourage inventiveness and help the team members form their own vision. This is exactly how efficient, innovative automation testing engineers are born and how the automation QA industry is constantly moving forward, powered by the creative thinking of its players.

How an automation team can change and evolve with time

A test automation team is not some fixed mechanism that performs tasks in a robotic manner and only changes when someone with more seniority tells it to change. A well-established, well-functioning automation team will transform and evolve naturally as part of its course. The question is which direction to choose for the transformation.

Perhaps, the most straightforward one is adopting new technologies to the mix to make testing more efficient and all-encompassing. For example, ideally, a senior-level automation QA needs to:

• Know two programming languages — an object-oriented one, such as Java or C#, and a scripting one, such as Ruby, Python, or JavaScript;
• Have experience with automation at different levels — UI, API, Unit, etc;
• Be familiar with different automation platforms — web, desktop, mobile;
• Have both the desire to manage a team and knowledge of how to do it;
• Be able to act as a visionary on a project, not just as a person to execute other people’s vision.

It’s also worth mentioning that not every automation QA wants or needs to assume a senior or management role, just as not every manual QA wants or needs to go into automation testing. However, with adequate leadership within the team and enough opportunities for improvement, each member of the team can locate and execute a comfortable growth trajectory.

Final thoughts

When done right, automation testing can dramatically improve the quality of the product while saving time and other resources in getting there. However, the key element and the most important aspect of an automation QA project is the automation team you’re working with. There are different ways to set up the automation operation at your company: hiring an in-house team, outsourcing the whole project, or choosing a combination of the two.

What matters the most is that each team member is there for the right reasons, knows the objectives of the project, and strives to become better at automation testing week after week. With that in check, and with talented management, the only way to go is up.