What’s your problem?
Use a hypothesis backlog to capture and refine your problems
As an Experience Designer, I get loads of info about opportunities to give customers a better experience, whether it’s user research, feedback from proxies such as Customer Support, best practices, anecdotal hunches and so on.
There were a few moments that made me realise I needed to track potential problems and improvements:
- When I introduced myself to someone in an office kitchen and casual chit-chat was quickly replaced with a list of things that could be better about the product I had just started working on
- Receiving a huge number of usability testing findings from a research piece that wasn’t specifically about the product I was working on. They were side findings (and good ones!) but each of them needed their own discovery work and wouldn’t fit in our backlog and I didn’t want to lose all the info
- Some people have knowledge about the known problems with a product, but sometimes it’s not listed or compiled somewhere and linked through to the research
I needed a way of packaging problems and opportunities so that I could tell the story to Product, then work together to get a priority. The product backlog is not a place to add in possible problems. These problems need their own space.
Even if these problems are not a priority right now, it doesn’t mean they won’t be a priority later, especially if we repeatedly see the same or related findings popping up again.
What I’m doing
I’ve created what I’ve called a ‘Hypothesis backlog’. It is a collection of opportunities to improve the product — some guesses, some validated by research. But likely, the prioritised ones need further discovery and problem validation.
It is not a backlog in the sense of a stream of work for developers or a roadmap for the Product (but could end up becoming part of these things if prioritised and validated).
Each ticket has a hypothesis about a problem. E.g. “We believe that the primary CTA doesn’t look like a button and this causes confusion” or it could be a ticket for a research finding e.g. “Research shows that people cannot find how to…”.
A problem hypothesis doesn’t have to be super scientific at this stage — it might just be a hunch or a guess or it could be a research finding. But framing it this way can help highlight the difference between a hunch (which needs further validation) and something proven by data.
The idea is that you list as much information about the problem, in one centralised place. Then, I sit with Product and go through these to see what might be worth spending time on.
In the ticket I include the following information:
Sources that tell us there is a problem — e.g. user research findings, other teams, best practice, etc. But don’t be afraid to put in anecdotal stuff too — later you can rate how confident you are that it is a problem (and a problem worth solving)
Tag who has mentioned the problem — I like to let people see the bigger picture if their feedback has come up before in research or otherwise. It let’s them see all the info and add their own notes and see that their feedback is useful! This hopefully will encourage more feedback
Links to related hypotheses, previous work, etc. useful for:
- Reuse of previous work (and understanding why it wasn’t implemented)
- Building a bigger picture — the relationships between problems might give you lots of insight about what it is you need to solve first, or if there’s a bigger overarching problem
Possible solutions, but…
This is useful for showing possibility or demonstrating conventions that are already set that you should reuse. But don’t worry too much about this part because at this stage it’s important to define the problem, and the impact of that problem, not the solution. If the hypothesis is prioritised for discovery work, you can explore solutions then.
This lets you work on the most valuable problems first. If you solve everything now, you won’t have focus. The quality of your solutions will erode and they might even become outdated by the time you even refine them. So don’t worry too much about it now!
Notes about tech, legal or other feasibility
- If you know something is a major blocker to solving the problem, it’s worth listing it. Maybe not everyone seeing the problem is aware of what’s stopping you from fixing it
- Equally, try to challenge these blockers! Are they really there anymore? Is there another way to get around it? Is it because of cost? Has someone else already solved the problem? Maybe if you build the picture, people might realise the problem is important enough to pay the cost for when you build this picture
- The idea is not necessarily to address everything now — the idea is for things you know you can’t do, it’s better that everyone who was concerned about the issue can see the action chosen — e.g. we can’t do it for legal reasons for another 6 months
Questions/doubts/things telling us it might not be a problem
It’s so good to state what you don’t know. Also make sure you ask questions of the people raising feedback. A lot of times users or customers or people internally will articulate the problem as a solution. You want to dig and dig (ask ‘why’ a lot) until you understand the core reason. Or, at least note where you think this digging needs to be done so if prioritised, you can look into those areas.
Benefits of problem hypotheses
- Prioritisation of problems becomes easier — allowing you to have regular catch ups with Product and other teams, to prioritise problems. There might be something that’s really worth working on, but no one has ever connected the dots together to tell the story
- Research findings are collated in a central place — instead of referring to separate findings documents, you can collate and connect the findings in one place
- Keep track of things that are coming up repeatedly — things might be a bigger problem than first expected, but if happens again and again, it’s worth revisiting
- Get a bigger picture — link up relevant info and people who are talking about the same thing and communicate why something that is seemingly urgent, cannot be tackled straight away (and challenge these reasons)
- Easy to link to and make visible “hey this has come up before — can you add your comments here”
- Highlights issues where you’re investing in research and seeing the same findings but not necessarily solving the issues. This can pique interest in solving a problem (or at least inform the design of research activities)
- Gives people a space to have their input recorded, that’s open and visible. They can then see the result of their input if the hypothesis leads to discovery and build work
- Creates transparency — if you can’t do something right now, or are deciding to park it for the long term, at least you can capture and communicate why
- Knowledge transfer — when someone new starts or when you have team members leave, this allows you to have a centralised place for this info, avoiding the process of digging through loads of documents. It also helps everyone have the same view of a problem rather than having to dig in people’s heads. This can help to mitigate corporate amnesia
- Context for your team — If the problem is prioritised and actually goes through to discovery, design, research, build etc. it is a one-stop-shop to take people through the why — hopefully everyone in your team has already contributed to it before e.g. even developers observing research
- Find contradictory information — sometimes different information will conflict and this can help compare
- Set yourself up for shippable increments — by focussing on the problem, breaking it down and thinking about what might be the smallest thing you could do to learn more, you’re already setting yourself up for small iterative design
- Effort vs reward — something may be a problem, but by talking through with Product, you can start evaluating what kind of value you will bring by solving it
- Identify small wins — some things will be small and quick to fix
Related reading:
Read Intercom’s ‘Great PMs don’t spend their time on solutions’ — I wanted to yell out ‘Preach!’ a million times as I read through
