Table Of Contents
I was working with a client a while back, a mid-sized SaaS team, about forty people across dev, QA, and product. Good people. Solid process. They had a regression suite with over 2,000 test cases and a team of six testers executing them every release.
And they kept shipping bugs to production. Not small ones either. Payment-related, data-loss-level stuff.
When I dug into it, the problem was obvious. They were testing everything with the same level of attention. The login page got the same coverage as the billing engine. A text alignment fix got the same rigour as a refactored API endpoint handling financial transactions. 2,000 tests sounds thorough. It wasn’t. It was effort in the wrong places.
Further, it was all about coverage. Not ensuring proper testing, but making sure everything was tested against every client. 2000 tests – 4 failed = great job!
That’s the story of most teams I work with. They’ve heard the term “risk-based testing” a hundred times. It comes up in interviews, retrospectives, conference talks. But when you look at what they’re actually doing, it’s either test-everything-equally or gut-feel-and-hope. Neither works.
This is Part 1 of a three-part series. Here I’ll cover what risk-based testing actually means and why it exists. Part 2 gets into the mechanics and Part 3 tackles the hardest part: getting people to actually do it.
What Risk-Based Testing Actually Means
Risk-based testing is a prioritisation strategy. Simple as that.
You make a list of risks. You score them. You test the high-scoring ones with more depth and the low-scoring ones with less. You adjust as things change. The formula is likelihood multiplied by impact. The higher both are, the more attention something deserves.
But here’s what separates actual risk-based testing from the version most teams do, which is basically gut feel: it is not about testing less. You go deeper where failure is costly, and you maintain minimum coverage everywhere else so you don’t accidentally ignore something embarrassing.
Some teams hear “risk-based” and think it’s permission to skip things. It’s not.
It’s permission to be deliberate about where you spend your finite time. There’s a difference, and that difference matters.
The Business Reality
Nobody says this out loud in sprint planning, but everyone knows it. You cannot test everything with equal depth. The product keeps growing. Features keep shipping. The test surface expands every sprint, and “just hire more testers” is not a strategy that scales. At some point, and it comes sooner than most teams admit, you have to choose what gets thorough attention and what gets a lighter touch.
Not all defects are equal either. A text overflow with zero functional impact on a settings page is annoying. A bug that duplicates payments is a major incident. I have seen teams spend three days testing a cosmetic change to a dashboard header while the checkout flow, which had been refactored the previous sprint, got a smoke test and hope. Treating these with equal rigour makes no sense. But without a framework for making different choices, teams default to treating everything the same because it feels fair. Fair isn’t the same as smart.
Agile cycles make this worse. Most sprints are two weeks. That is not enough time to test every feature of a modern application from scratch. And as the product grows, the gap between what should be tested and what can be tested gets wider. You need a principled way to decide what happens first when time runs out.
And it always runs out.
There’s also the stakeholder angle, which I think is underrated. When you report testing in risk terms, you give people information they can actually act on. Instead of “we ran 847 tests and 12 failed,” you can say “we’ve tested the top 10 risks. Two remain partially mitigated. Here’s what that means for the release.” That’s a conversation about business trade-offs. That’s QA adding value beyond “we found some bugs.”
The Traps
Most teams fall into one of two traps. Both end in the same place: critical bugs in production, QA getting blamed, and everyone wondering what went wrong.
Trap one: testing everything equally. On paper, this sounds thorough. In practice, it’s how you end up with 200 test cases that all pass while the one thing that’s going to cause a P1 incident next week has ten cases covering it and fifty covering a feature nobody uses. I am not exaggerating. I have seen this first hand. The team was genuinely confused about why their “comprehensive” test suite kept missing things. The answer was simple: coverage isn’t the same as focus.
Trap two: gut-feel prioritisation with no documentation. Someone on the team, usually the most experienced person, decides what matters. Testing follows that path. And honestly, this can work for a while, surprisingly so. Experienced testers have good instincts.
But again it doesn’t scale.
When that person leaves, the knowledge walks out the door. When you need to explain to a stakeholder why something wasn’t tested, you’ve got nothing to point to except “well, Sarah thought the risk was low.” When someone challenges your priorities, you’re defending vibes. And when pressure mounts before a release, gut feel buckles under the weight of whoever shouts loudest. I have watched a product manager override a QA lead’s testing priorities because they were louder in the room, not because they were right. With no documented rationale, there was nothing to push back with.
The Misconceptions
“We need a fancy tool to do this properly.” You really don’t. As much as I am not a fan of spreadsheets, they work fine. I have seen teams spend three months talking about evaluating risk management platforms and then never actually do a risk assessment. The tool is not the point. The thinking is the point. Start with a spreadsheet. You can always upgrade later if you need to.
“Risk assessment is a one-time activity.” It isn’t. Risk changes every sprint. New features, production incidents, refactored code, changed usage patterns. A feature that was low risk six months ago might be high risk now because your biggest customer started using it daily, or because someone rewrote the service layer underneath it. Revisit regularly. That’s the whole point.
“This is just QA’s job.” This one frustrates me the most, because it misses the entire point. Developers know where the code is fragile. They know what was rushed, what has technical debt, what’s held together with workarounds. Product knows what the business can’t afford to lose. Support knows what users actually complain about. Ops knows what keeps breaking at 2am. QA should facilitate the conversation, sure. But if QA is the only voice in the room, you’re building your risk picture with one eye closed.
“Following risk-based testing means we’ll never miss a critical bug.” No. It improves your odds. You’re pointing your effort at the areas most likely to fail and most likely to cause damage. That’s better than spreading effort evenly, but it is not a magic shield and sometimes you’ll still get it wrong. The goal is to get it wrong less often, and to be able to explain your reasoning when you do.
“Low risk means skip it.” Low risk means less testing, not zero testing. A quick sanity check. Confirm it loads, confirm the happy path works, move on. Skip it entirely and I promise you, something embarrassing will eventually make it to production. It always does.
Why This Matters for QA’s Role
There’s a bigger picture here that goes beyond test efficiency, and it’s something I keep coming back to across multiple articles now.
When QA operates without a risk-based approach, you’re just ticking a box. Stories come in. You test them. You report results. You’re reactive, dependent on what others give you, and the first person everyone looks at when something breaks. That’s a frustrating place to be. We have probably all been there.
When QA operates with a risk-based approach, you become someone who actually influences delivery decisions. You’re identifying risks before they become problems. You’re having conversations about trade-offs with product managers and engineering leads. You’re shaping what gets released and when, not just validating it after the fact.
That shift changes how the rest of the organisation sees you. It changes the conversations you’re invited to. And to be sure, it makes the job a lot more interesting than running the same regression suite every fortnight.
What’s Next
Understanding why risk-based testing matters is the foundation. But knowing it’s a good idea doesn’t tell you how to actually do it.
In Part 2, I’ll walk through how to identify risks using both inside-out and outside-in analysis, how to score and prioritise them, how to map testing effort to risk tiers, and how to keep it all current as the product evolves. There are templates too, same ones I use with clients.
Then in Part 3, we tackle the hardest part: getting buy-in and making it stick when the pressure is on. That’s where most teams fall over, and it’s not a technical problem.
The Take
Risk-based testing is a prioritisation strategy. You focus effort where failure is most likely and most damaging. The formula is simple: likelihood multiplied by impact.
It exists because you can’t test everything equally, not all defects are equal, and delivery timelines keep getting shorter. Ignoring that leads to wasted effort and escaped defects.
The traps are testing everything the same way, which feels thorough but isn’t, and making undocumented gut-feel calls that can’t survive a personnel change or a heated meeting.
Done well, it changes QA from the team that finds bugs to the team that manages quality risk. That’s a different job, and a much more enjoyable one.
Part 2 covers the how.
A Note on Context
Every business and every project is different. What works in one place won’t work in another, and that’s the point.
Nothing here is meant to be a step-by-step prescription. It’s guidance, drawn from my own experiences, and deliberately kept general to avoid pointing fingers anywhere.
Take what’s useful, ignore what isn’t, and adapt it to your own context. Or as Joe Colantonio always says: “Test everything and keep the good.”
