Traditionally, testing has been the last step in the development of the system before completion. In principle, therefore, it could not have been very agile because comprehensive testing requires a functional version of the entire system.
However, new development models question this thinking, and few projects have the opportunity to set aside sufficient time and resources for testing at the final phase of the project. So we definitely need testing along the way, but in practice there are various challenges.
Challenges of agile testing
Comprehensive and high-quality system testing requires sufficiently precise requirements for the basis of test cases and possible test automation. In agile development, however, the requirements become more precise or even change along the way. User stories are rarely accurate enough at the beginning of a project to define all test cases. With the completion and review of sprints, the team's understanding of the solution will change or at least become more precise, and consequently the tests will need to be refined as the project progresses.
Testing experts should therefore be involved in important meetings, such as sprint planning and review. It is also important that they understand both the key features for the customer and the non-functional requirements (such as security) and know how to target testing efforts to these. But their role is no longer to perform all the testing.
The development team implements new functionality for the product in each sprint every week or two. How can these be comprehensively tested so that the team can confidently demonstrate new functions? This is a key issue to consider even before the project is up and running.
According to the principles of agile testing, the entire team tests and controls quality whenever outputs are generated. Developers run unit tests before coding begins and test the stories as they are completed. Ideally, these should be as automated as possible. The Product Owner defines the acceptance criteria for user stories, and also tests against the criteria once stories are implemented. The customer performs acceptance testing once a suitable release is ready in the production environment.
The broader role of the test team
When in agile development the whole team is responsible for quality, the role of the testing team (or rather testing experts) is very different. Their main job is to support the rest of the team so that testing can progress with other development. In practice, this may include, for example:
- promoting and ensuring a testing perspective (acceptance criteria) when writing user stories
- deployment and support of test automation and other testing tools
- assisting the rest of the team in quality assurance tasks
Effective testing and its automation today requires strong knowledge of both the tools used for it and the system being tested. For example, if new interfaces are built into systems, these should be tested separately and not just through the applications that use them.
However, not all developers or product owners may have sufficient expertise, and the help of testers is worth their weight in gold. This should also be kept in mind when teams are being resourced. Unfortunately, in practice testing expertise is often severely underrepresented.
The importance of automation
Why is automation important in agile testing? The answer is, in the end, quite simple. Iterative development produces new functionalities on top of a foundation that may grow in each iteration but is (usually) unchanged. It is important that tests on this foundation be automated as far as possible and performed during all future iterations.
If automation cannot be built, the amount of testing required will become unreasonably large. Not only does the new functionality need thorough testing, but the foundation also needs to be tested to ensure that changes have not affected its external performance. It is also worth remembering that the refactoring associated with agile development increases the need for regression testing.