If you’ve spent enough time in your Quality Assurance, perhaps with a traditional title like “QA Engineer” or “QA Analyst”, then you have probably heard the following directive: As testers, we must tell a compelling story to our stakeholders. The context-driven testing community believes this to be paramount to anyone serving in the testing role.
But, what does this really mean? Are we just talking about a scripted checklist here? Are we just trying to sound elite? Is this just some form of covering ourselves to prove we’re doing our job?
Well, none of the above actually. The purpose of doing this is to continually inform our stakeholders in order to increase their awareness of potential product risks, so that ultimately they can make better business decisions. QA is not the job of a single individual, but rather a group effort that involves both the team as well as product management and business aspect outside of the immediate development group. This larger group needs to be informed in order to make the proper product decisions.
We can use various methods to tell them about what was and what was not tested. First we must agree on the meaning of the words “compelling” and “story,” then we’ll dive into the logistics of how to deliver that message to our stakeholders.
Editor’s Note: The complete post of this blog, which includes visual models and more can be found at PixelGrill.com. Also, throughout this piece, I am also going to use the term “Product Management.” When I say that, I am referring to the people who end up doing the final risk assessment and make the ultimate business decisions as it relates to the product (more about that here from Michael Bolton). This may involve the Product Owner on your team, or it may involve a set of stakeholders who are external to the team.
The word “compelling” can seem a bit ambiguous and its meaning can be rather subjective, since what is compelling to one, is not so to another. What convinces one person to buy a specific brand, does not convince the person right next to them. However, regardless of your context, we need to set some guardrails around this word. The reason for doing this is to remain in line with the community’s endeavor to establish a common language so that we can properly judge what qualifies as ‘good work’ within the realm of QA testing, and in this case specifically, how good one is at telling a compelling story as testers.
Yes, you as a tester should be constructively judging other testers’ work if you care about the testing community as a whole. We cannot do that unless we’re armed with the right information. So first, let’s take a very literal view, and then move forward from there:
“Compelling” as defined by Merriam Webster:
(1) very interesting : able to capture and hold your attention.
(2) capable of causing someone to believe or agree.
(3) strong and forceful : causing you to feel that you must do something.
The information you present should carry with it hallmarks of these three definitions, regardless of the target stakeholder’s role within the company.
Let’s elaborate, specific to the context within a software development environment.
- (1) Interesting: As a tester, by being a salesperson of the product, and a primary cheerleader for the work the team has done, I am satisfying the first criteria. I know all the ins and outs of the product, thus being a subject matter expert gives me the ability to speak to its captivating attributes in order to draw my stakeholders into the discussion. (This also involves knowing your stakeholder, which I could write an entirely separate article about, explaining how you tailor your story for specific stakeholder roles within the company – more on that later).
- (2) Cultivate Agreement: As the tester for a given feature, you are aware of an area’s strengths and weaknesses. It is your job to take multiple stances, and defend them, be they Pros (typically in the form of new enhancements or big fixes) or Cons (typically in the form of product risks).
Just like the defense given by an attorney in their closing arguments of a trial, so too should you defend your positions, regarding the various areas of the product that have changed or are at risk. Since you are informing on both what you did and did not test, then you can aid much better in joint risk assessment with Product Management.
This is how testers influence the product development process the most; not in their actual testing, but in their conversations with those who make the business decisions when telling the story of their testing. Give your opinions a solid foundation on which to stand.
- (3) Take Action: All information that you give to stakeholders should support any action items they may need to take based on that data, thus you should be a competent professional skeptic in your testing process so that the data best leads Product Management toward fruitful actions.Your feedback as a tester is instrumental in well-intentioned coercion, or since that’s typically a negative term, let’s call it positive peer-pressure. A Product Owner should be embedded, or at least in constant communication with, the scrum team thus any actions that arise from this information will be of little or no surprise to the team.
On that note, surprises generally only occur when the above types of communication are absent which of course is not just limited to testers. Either user requirements are ambiguous or not prioritized (by Product Management), or perhaps there are some development roadblocks and test depth is not made tangible (by the Scrum Team).
I use these three elements as a heuristic to prime the thinking of our stakeholders, so that they can make smarter and wiser product management decisions for the business.
Becoming A Storyteller:
It might seem obvious to say, but the best storytellers always include three main parts: beginning, middle and end. More specifically, characters, conflict and resolution.
In a well-structured novel, the author typically introduces the reader to a new character for a period of time, for the purpose of initial development for the audience.
Soon, a conflict arises, followed by some form of conclusion, perhaps including resolution of some interpersonal struggles as well. In testing, we want to develop the characters (feature area prioritization), overcome a conflict of sorts (verify closure of dev tasks and proper bug fixes based on those priorities) and come to a conclusion (present compelling information to Product Management about your testing). Just like an author describes a character’s positive traits as well as their lacking characteristics, we too should be sure that our testing story includes both what we did test and also what we did not test. Many testers forget to talk about what they did not test, and it goes unsaid, which increases the risk gap.
However, unlike a novel where a cliffhanger ending might be intentionally crafted in order to spur sales of a second book, omission of information to our stakeholders should never be intentional, and is not part of the art and science of testing. If this is done, the human brain will naturally fill in gaps with their own knowledge which may be faulty, or worse, make assumptions which can become fact if left unchecked for a long enough amount of time.
The problem with assumptions is that they are grown within a hidden part of the brain, only knowable to that individual and typically do not expose themselves until it is too late.
Leave as few gaps as possible by becoming a good storyteller. It can be dangerous when a tester becomes “used to” the mental state of not telling a story; believing that their job is simply defined by their test case writing and bug reporting skills as they currently exist. As testers, let us not be so limited or obtuse in our thinking when it comes to exploring ideas that help us become a better tester, otherwise our testing skill-craft risks being destined to remain forever in its infancy.
The Content Of The Testing Story:
Now, no matter how good a salesperson you might be, or how convincing and compelling you may sound to your various stakeholders, your pot still needs to hold water. That is to say, the content of your story must be based on solid ground. There are three parts to the content of the testing story that we must tell as testers: product status, testing method and testing quality.
- Product Status: We should tell a story about the status of the product, not only in what it does, but how it failed or how it might fail. This is when we report on bugs found and other possible risks to the customer. This report should be value-add, in that it should not contain a lot of edge cases, but rather take into account the real risks that matter to the clients.
- Testing Methods: We should tell a story about how we tested the product. This involves using oracles, which are heuristic principles or mechanisms by which we recognize a problem. More on that here. We can use oracles to recognize problems and createwhich are heuristic principles or mechanisms by which we recognize a problem. More on that here. We can use oracles to recognize problems and product coverage outlines that help visualize our testing process. Which heuristics are you using to do your testing and why are those methods valuable? How did these models affect product operations as well as lead to observations? You’ve spent all this time talking about what you did test, but did you also talk about what you did not test or what you intentionally are not planning to test? Testers tend to do a better job as talking about what was done, but not what wasn’t done as a part of the testing process.
- Testing Quality: Finally we should talk about how good our testing was. What were the risks and costs of testing? What made your testing harder or slower? What roadblocks came up that required the testing effort to pivot? Based on your testing process, how testable or not did you find the product to be? What issues did you present as testers and how were those addressed or alleviated?
All three of these elements help us to make sure the content of our testing story is not only sound but also durable in order to hold up under scrutiny.
The Logistics of Telling the Story:
So, what is our artifact, that we, as testers, have to show for our testing at the end of the sprint? No, it is not bug counts, dev tasks or even the tests we write.
Developers have the actual code as their artifact, which is compelling to the technical team, given it can be code reviewed, checked against SLAs, business rules, etc. As testers, traditionally our artifact has been test cases, but as a profession, I feel we’ve missed the mark if we think that a test case document is a good way to tell a compelling story. Properly documented tests may be necessary in most contexts, but Product Management honestly does not have the time to read through every test case, nor should it be necessary. Tests cases are for you, the testing team to use for the purposes of cross-team awareness, regression suite building, future refactors, dev visibility, etc, while it is actually the high-level testing strategy that really provides the most bang-for-buck value add for stakeholders in Product Management.
As far as the actual ‘how-to’ logistics of the situation, there are multiple options that testers should explore within their context. Since humans are visually-driven beings, a picture can say a thousand words, and the use of models provides immense and immediate payoff for very little actual labor. Now that we’ve established criteria for how to make a story compelling, and what the content of that story should be, let’s take a look at the myriad of tools at your disposal that can help with the logistics of putting that story together.
Humans are visual beings, and our brains absorb information much faster in this more tangible form than any other method. We talk a lot about using tools to visualize your testing, which can be done using explicit test strategy models that have already been created and exist within the community. These help inform our thinking about how to examine a particular feature, since we cannot “cast light on the status of the product” if we do not have a capable tool belt with which to work. The most accepted and commonly used model in the context-driven testing community right now is the HTSM, Heuristic Test Strategy Model by James Bach (XMind Link). This is a model that can greatly broaden our horizons as testers and help product management make much smarter and more informed business decisions.
We have other tactics at our disposal like Decision Tables and Testing Mnemonics that can help critical thinkers fill in the gaps that they may have missed. This all ties back into having a genuine desire for learning so that we can increase our skill-craft as testers. If interested, you can read more in an entry I wrote for my blog PixelGrill, The Improvement Continuum.
Tailor Your Story:
Finally, there are multiple ways to tell the same story, and your methods should change depending on your audience. For example, we should not use the same talking points with C-level management as we would with our Product Owner. Since the relationship to the product is different for each role within the company, then the story should also be different. You would use the same themes obviously, but your language should be tailored to best fit their specific perspective as it relates to the product.
In this video, Talking with C-Level Management About Testing, Keith Klain discusses how different that messaging should be, based on your target audience.
My favorite quote from that video is when the interviewer asks Keith, ‘How do you talk to them about testing?’ to which he replies, ‘I don’t talk about testing.’ He then explains how we can discuss testing without using the traditional vernacular that can sometimes be unintentionally misleading.
Being aware of your audience should influence not only how you test, but how you talk about your testing. I might be compelled to write another blog post specific to this topic; that is, if there’s enough interest in how to mold the testing story based on the various roles within your stakeholder demographic.
It is common for Product Management and development teams to be two completely different pages when it comes to managing customers expectations.
Developers and testers can lose sight of the business risks, while product owners and VPs can lose sight of the technology constraints. Ultimately, it is the job of Product Management to make the final call for deployment of new code, while our job as testers is to inform those management folks about any potential product risks related to the release. This is mentioned in the opening, but it is worth highlighting again; this Developsense Blog post by Michael Bolton explores that tangent.
Again, the purpose of testing is
“to cast light on the status of the product and its context in the service of our stakeholders.” – James Bach.
Testers tell a compelling story, but at the end of the day, your story should roll up to that.
If it does not, then reevaluate: Is the information you’re telling for your benefit, or your stakeholders’?
Be professionally skeptical and ask yourself questions:
- Is this worth sharing?
- Have I made it compelling enough to drive home my main message?
Many Product Owners have not had any interest in what their tester documents, because it has traditionally been of little value to them. Don’t use a Product Owner’s lack of desire as an excuse to stunt your own growth as a tester.
Get good at this, and become a more responsible tester. While any failing caused by apathy is the responsibility of the entire team, testers are at the helm of testing and have the power to change it for the better. is the responsibility of the entire team, we are at the helm of testing and have the power to change it for the better.
What’s your experience? I’ve made the case for us, as testers, needing to tell that story. At PixelGrill.com you can find other ideas and techniques to get started putting this into action.
How do you currently do this? Share with us here, hit me up on Twitter @connorroberts or comment on the complete post at PixelGrill which parts interest you most, from the material here, or your own discoveries in the comments.