Deployment strategies: A business guide
Several different models and deployment strategies have been emerged to meet the specific needs of different business types.
There is no general consensus that the work of Quality Assurance (QA) engineers should be a necessary part of the software development process. Some companies are convinced that software developers should be the ones who write code and provide QA by testing their own code; therefore, there is no need for the distinct role of QA or Tester. At the same time, Google approaches this issue from the opposite angle, giving testers ‘an enhanced responsibility of being the authority on product excellence’.
Let’s discuss the role of a QA engineer to highlight their skills and responsibilities on a given project.
Types of testing
Tests are divided into two groups: functional and non-functional. QA engineers have proficiency in all test types.
Not all tests are performed by QA engineers alone; sometimes they need to collaborate with other specialists. For instance, unit tests are written by software developers to ensure a particular software module works as expected; usability testing is provided with the help of usability specialists; and security specialists are often involved in security testing.
It would be a mistake to expect that unit tests and integration tests, even when combined with a test-driven development (TDD) approach, can replace all other types of testing. QA engineers test not individual software components, but the whole system by following a precise test plan that covers all types of tests. The most valuable skill for QA engineers is the ability to put themselves in the users’ shoes, analyse where the software product could be broken, and find ways to address these issues by developing test cases based on user flow.
Types of QA engineers
Testing can be implemented by means of manual and automated methods. Therefore, all testers are divided into QA manual engineers (or manual testers) and QA automation engineers (automation testers).
Manual testers combine product knowledge and QA domain expertise, focusing on improving the final products and end-user satisfaction. Automation testers build the frameworks and packages required to implement QA automation.
Some companies might consider forgoing the Manual tester role, as this function can be provided by various specialists, such as project managers, team leaders, data analysts, or software developers. However, it would not be done in a systematic way. To ensure the high quality of a final product, the testing process must be systematic, based on a detailed test plan. The leading role in this process is played by QA manual engineers.
Also, it would be a gross mistake to task automation testers with conducting manual testing by themselves. No doubt, they understand manual testing methods and techniques, but their strength is automation, as they would have mastered programming skills to avoid all routine manual activities.
A Test lead or QA lead is responsible for the coordination of the test engineer team and QA process setup. Further, the test lead is expected to find ways to optimise testing flow and automate processes that consume a lot of test resources and time. Manual methods are still required, and, in some cases, they are the only available option, but their use is expected only when automation is technically impossible or assessed as overly expensive.
In addition, we should specify what can be tested. It might be the business logic of software products, UI, and data. Data testing is the responsibility of a QA data engineer, who is responsible for data quality and validation. Apart from proficiency in commonly-used testing tools and techniques, these specialists need deep expertise in SQL, Python, or R programming languages.
On small projects, some ‘hybrid’ QA engineer types can appear, i.e. QA/BA or PM/QA specialists, who combine the roles of quality assurance engineers and business analysts or project managers.
Why software developers should not test their own code
So, why is it a mistake to expect software developers to write test cases for their own code? There are five reasons:
- The process of writing the tests requires time. So, developers must choose whether to write tests or produce new code. The presence of a QA engineer in a software development team allows programmers to concentrate on their core functions and be more efficient in achieving the overall project goals.
- Developers won’t draw up a detailed Test plan, without which it is impossible to achieve a consistent and systematic approach to testing software products.
- Developers don’t always have a holistic vision of the software architecture, as they often work only with one part of the application. Testers examine the entire system, not its separate pieces, to find the common bottlenecks and errors.
- There is a difference in the approach of testers and developers, which prevents them from reviewing their own code objectively and testing the system to find the failures rather than to obtain a positive result.
- To ensure product quality, QA engineers act from the users’ perspective. They focus more on verifying adherence to existing requirements by examining user flows and application use cases.
QA engineer responsibilities
The main goal of QA specialists is to ensure an ideal testing and development environment using the best testing practices to guarantee product excellence.
The software testing life cycle consists of the following stages: requirements analysis, test planning, test case development, test environment setup, test execution, and test closure activities, which include quality reports, suggestions for improvement, and, in some cases, customer support.
Requirements analysis
First, QA engineers need to analyse the requirements for testing a software product and check them according to five key attributes: completeness, clearness, correctness, consistency, and testability. If the list of requirements doesn’t meet these attributes, they cooperate with the project manager to discover missing or inconsistent components.
Also, testers should be involved at each stage of a proposal for a new feature, so they can review it and provide recommendations about its implementation in testability.
It’s always cheaper to prevent defects rather than being forced to fix them at a later stage. A mature quality assurance process must include the rule ‘No release before testing’.
Test planning
The result of the whole testing process heavily depends on the accuracy of the selected testing instruments and methods. As the most critical stage of testing, planning includes defining the scope and objectives of the experiment; estimating test effort and resources; and specifying the testing environment and tools needed, resource allocation, test techniques to be followed, and the risks and mitigation plan. Test lead or project manager in collaboration with QA engineers documents all these points in the Test plan, which should reflect the principles of the General Test Strategy and Test Policy, the document that represents the testing philosophy of the company as a whole.
Testing environment setup
The project manager or test lead cooperates with the project architects and DevOps engineers to define the initial test environment, verify the availability of the resources required, and ensure that people responsible for the testing environment configuration are committed to its creation.
Test case development and execution
The test case is a set of conditions and expected results that allow a QA team to determine whether a tested system satisfies requirements and works properly, using either manual or test automation methodologies. The lead role in this stage is played by the manual QA. They should create a test scenario based on possible user behaviour inside the system, which requires deep QA expertise, an understanding of the potential user’s mindset, and creativity in using testing methodologies and tools.
Quality reports and suggestions for improvement
As quality experts, QA engineers are expected to contribute a lot to the improvement of the final product, end-user satisfaction, and the overall production process. Preventing production outages is the main task for testers. Almost 50% of testers note that submitting quality reports and analysis of production data is an essential part of their role in a team.
Customer support
QA engineers serve as a bridge between the customer support team, who deals with end-users, and the production team. Software developers have neither desire nor time to deal with the preliminary requirements analysis. Thus, testers create testable customer journeys, analyse user feedback to find the cause of product imperfections, and collaborate with the development team to fix any issues they uncover.
How does a product work without proper testing?
The reality is cruel – It doesn’t work! Remember Windows Vista? In 2007, Microsoft placed its highest expectations on this release. The company invested $500 million in a marketing campaign and predicted that half of the current users would migrate to the premium edition within two years. But even their loyal customers couldn’t tolerate the enormous quantity of problems in compatibility and performance. Its primary competitor, Apple, used this production fail in its ad campaign for Mac devices that caused not only commercial but reputational harm.
Today the impact of negative user feedback is even more significant due to the fast spread of information through social media. Therefore, it’s better to postpone the product launch and ensure high quality rather than release something with obvious defects. There are no second chances for products in the real world.
How to ensure project quality?
It is essential to remember that the final quality of a released product is the responsibility of each member of the development team, including the product owner. The product owner should be fully engaged in the software development and testing processes and control quality at all stages.
Test plan and test coverage are the main indicators to check quality during the development stage. The first important matter is requirement coverage, which demonstrates how many requirements have been covered by test cases and determines the particular requirements that should never be tested. These test cases focus on the user expectations of the application.
It is crucial to ensure that all lines of code are executed across all existing test cases. Test management systems such as TestRail or TestLink allow for history checks on all test cases, plans, and runs, and calculate a test coverage level.
Testing effectiveness can be measured by some key indicators:
- Defect density illustrates the number of defects per thousand lines of code. The result of one defect per thousand lines of code is generally considered a sign of good project quality.
- Defect reopen ratio is used to evaluate the quality of software development and defect fixes. If this indicator exceeds 0.2–0.3, we can assume problems in project quality as a whole. In this case, it can be useful to trace the workflow of a defect life cycle through bug tracking systems such as Jira, Mantis, etc. This will provide an understanding of whether these problems are caused by inexperienced testers or software developers.
- UAT defect leakage indicates how many defects were overlooked and went to production, or user acceptance testing (UAT). It is expected that defects found during UAT will be less than 0.1.
There are only three ways to fix the project quality at this point: change a Test lead, hire a new team of more experienced testers, or replace the IT Solution Provider.
The tester-to-developer ratio
A frequently asked question is what tester-to-developer ratio is optimal for the development team, to which there is no definite answer. The structure of a team depends on the type of solution, industry, safety requirements, and many other factors. For example, projects in the healthcare industry can require even 15 or 30 testers to one developer. On the other hand, start-ups often have no budget for testers, teams involved in commercial off-the-shelf (COTS) software development usually have 1:1 ratios, and other companies focus on developers. Thus, ratios can be 1:2, 1:3, or even 1:5. In practice, one tester per three to five developers is a reasonable ratio for business solutions.
Common advice is to focus on the number of quality-related activities needed to achieve customer satisfaction. The higher the risk level, the more testers should be on a team. And still, we should keep in mind that one highly capable tester might be ten times more effective than less experienced colleagues. Here are some examples of how world-famous software giants organise their testing processes:
- Amazon’s tester-to-developer ratio is roughly one test engineer to seven developers. But as soon as the company notices a decrease in customer satisfaction, Amazon increases its testing efforts.
- According to the book Microsoft Secrets, the company employs a 1:1 or at least a 2:3 tester-to-developer ratio.
- As for Google, nobody knows exactly their tester-to-developer ratio. But, for example, testing changes to the search engine are organised in four stages:
- Testing by dedicated testers.
- Testing on a crowd-testing platform.
- Testing by Google employees in their daily work.
- Beta testing on a small group of end-users.
Moreover, every single software engineer is expected to test his own code, according to the book How Google Tests Software.
Requirements for a capable QA engineer
‘Is it really hard to find a good QA engineer?’ This question is heard frequently when involving testers in the project team, but underestimating their responsibilities can be a critical mistake. Finding an effective tester is not an easy task. Let’s look at a list of requirements for an experienced QA engineer.
1. Proven competence
Generally accepted professional certifications for a QA engineer include ISTQB® (International Software Testing Qualifications Board), provided by the International Software Quality Institute, the Certified Associate in Software Testing (CAST), and the Certified Software Tester (CSTE) certification from the Quality Assurance Institute. These organisations offer several levels of certification, starting from the beginner to the advanced or expert level. Another popular source for certificates is the International Institute for Software Testing (IIST).
Apart from these, Scrum and Agile certifications are also worth having for QA engineers on any software development team that uses Agile methodologies.
For some projects, another plus is if a QA engineer has certificates in software development or cloud solutions like AWS, Microsoft Azure, or Google Cloud.
2. Communication skills
Communication is a vital component of a successful product launch. 98% of testers evaluate communication skills as ‘important’ or ‘very important’ for a proficient tester, according to the State of Testing Report 2018. Quality engineers have to interact with product managers, development teams, end-users, and others to ensure product excellence during all software development stages.
3. Technical breadth
A talented QA engineer needs to know a little bit of everything. Good working knowledge of a wide range of contemporary technologies and emerging tools helps to enhance the delivery of quality assurance testing services. A high level of technical proficiency gives an understanding of how a software product works and where its breaking point may be.
The fundamental technical skills testers need in their job are the following:
- Testing techniques and approaches
- Functional automation & scripting
- API testing
- Security testing
- Performance testing
- Understanding of the software development life cycle
- Knowledge of databases, web, and mobile technologies
- Awareness of software architecture patterns
- Cloud platforms
- DevOps and understanding of the CI/CD process
- Programming skills
- Ability to work with documentation and requirements
4. ‘Test to break attitude’
There is a difference in mindset between software developers and testers. The first have a desire to build, whereas testers must find a way to ‘break’ the software product. However, the fact is that a tester only finds and reveals the things that the software developer has left broken. So, it’s better to say they have a hunch where to dig to discover breakage.
5. Domain expertise
It is always an added advantage when the QA engineer is a domain expert and understands the software product and business objectives. Industry experts need less time to find the breakpoints and advise the best practices on eliminating them.
6. Analytical thinking
The ability to analyse requirements for testing, investigating problems, finding the cause of bottlenecks in a software product, and providing recommendations for its elimination are the essential responsibilities for a tester. Another critical task is the identification, assessment, and mitigation of a software product and project risks.
Testers identify the hidden errors by analysing the solution and generating all possible scenarios of its use as a potential customer. Any mistake leads to the loss of audience and money. That is why a tester’s mission is to avoid the dreaded outcome of failed software or the release of a buggy product.
7. Seniority level
QA automation engineers have the same seniority levels as software developers: junior, middle, and senior. As with QA manual engineers, it is rare to find senior manual testers: they often acquire programming skills and become QA automation engineers or move up to management positions such as test lead, product manager, project manager, business analyst, customer experience managers, etc.
A project has little chance of success if testing processes are entrusted to junior QA engineers. Time is needed to gain practical experience and enhance technical breadth. Therefore, an efficient tester needs at least three years of relevant experience.
How to build an effective QA team?
- Rely on experienced QA engineers. High seniority level, certificates, domain expertise, technical breadth, and excellent communication skills are key to project success.
- Consider the tester-to-developer ratio to ensure that all test cases are completed within their initially scheduled timeframes.
- Be engaged in all processes that directly impact product quality.
- Build blended QA teams. A reasonable approach to ensure product quality and decrease the risk of failure is to combine internal QA practices with representatives from an IT solution provider.
Conclusion
As we can see, the importance of the QA engineer in the software development life cycle is quite conspicuous. Despite this, there are still companies that see testers as low-skill specialists responsible solely for searching for bugs after the development phase is over.
However, recent trends demonstrate that many companies adopt the best quality assurance practices and widen the responsibility of test engineers by turning them into product quality experts. The efficiency of this approach is demonstrated in the final quality of the released product and the level of customer satisfaction.
Hire the experienced software development team to ensure your project success and the high quality of your product.
- Author:
- SSA Group Team