A growing number of web and mobile technology projects are adopting an agile approach to development. One of the benefits of following the agile method of delivery is a faster time to market.
When the push is on to deliver to the customer at a faster pace, the pressure is also on the QA team to execute testing at a faster pace. How does this affect the traditional QA testing practice of creating detailed test cases and executing these test cases after development is complete? Exploratory testing provides value-added testing and allows the QA team to have confidence in delivering a quality product to the client.
What is exploratory testing?
James Bach is a software tester, author, trainer and consultant. He is a proponent of exploratory testing and the context-driven school of software testing and is credited with developing session-based testing.
In his article at satisfice.com, James defined exploratory testing is any testing to the extent that the tester actively controls the design of the tests as those tests are performed and uses information gained while testing to design new and better tests.
In other words, test design and test execution at the same time.
Where does exploratory testing fit?
Manual regression testing is tedious. Just ask a QA analyst that’s been part of any software regression test cycle. Executing a user path, using the same inputs, will get the expected results. Repeating these tests is mind-numbingly dull.
When automation has been established, the QA team can focus on intelligent testing. Exploratory testing encourages the application of critical thinking to the way a test is executed. It also challenges us to think “outside the box”.
Because it is test design and test execution at the same time, it allows a QA analyst to use their creative energy as well as be intellectually stimulated.
How do I test it if I don’t have defined steps and expected results?
Let’s think about how Columbus discovered America. He took three ships, left Spain and headed west until he reached what he thought was the East Indies. Instead, he discovered another continent. He had an idea of where he wanted to go but the result was not what he was expecting.
Exploratory testing is a discovery on how the application actually works. It is dependent on a QA analyst’s experience, ability to create effective test scenarios, and flush out meaningful defects – those defects that get fixed and aren’t pushed into a backlog until a customer discovers it in your live production environment.
How can you implement exploratory testing into your current testing process?
Explain the approach to the stakeholders and QA team
Arrange information sessions with the project stakeholders to understand the concept of exploratory testing. These can alleviate coverage concerns and corrects the concept of rogue testing.
Facilitate mind mapping sessions with the QA team to help them think creatively. You can also include members of the development team so that the QA team can understand implementation and expected results.
Put it into practice
Assuming you have a regression automation suite in place, create a formal test plan approach to combine Automation and Manual Exploratory testing during a pre-release regression test period
Identify the remaining regression manual tests that automation does not execute.
Create a matrix for level of test coverage against functional areas of the application.
Craft high level exploratory test scenarios in your chosen test case management tool. Steps will not be defined but rather statements “Given these conditions, when I do this, then this happens”.
Run automation suite to provide cross browser coverage. Utilize automation scripts to cover devices as well.
Execute the high level test scenarios. Record the results in the high level test scenarios, noting any issues found or any user flows or execution paths that came to the tester’s mind
Provide execution completion results in a status update. Include the percentage of coverage of key components based on your test matrix, number of scenarios executed and defects found as a result of the execution. The status you provide should reflect the coverage rather than x amount of test cases executed.
Organize Ongoing Audits and Maintenance tasks
To keep current with our test coverage and with ongoing development, set up on-going sessions for auditing release deliverables. Perhaps there are new features that require and automated test rather than an exploratory test. Create and maintain the exploratory test scenarios based on these audits. Revisit the exploratory test scenarios to identify any tests that could possibly be automated.
Exploratory testing can be a breath of fresh air to the QA analyst that has been executing step by step test cases for the lifetime of their quality career. Thinking like a user and exploring all the ins and outs of an application can uncover functional defects as well as user paths that developers or product owners may have missed. Testing can be more valuable and establish client confidence. It’s a win-win situation for all.
The Quality Yardstick 