Webinar Series Part 1 of 4: Hey, What’s Your Problem?

In product management, we often get presented with solutions:

Sally wants to take a picture.
John needs to call a cab.

Unfortunately, simply building these solutions means we sometimes miss relieving real pain for our users.

At my current company, Globant, I’ve created a curriculum and workshop series to help clients and internal team members learn a different way of approaching product development. To get the word out, we created a four-part webinar series to illustrate what you might learn in the workshops. The first one, Hey, What’s Your Problem? discusses ways to approach product development by focusing on user problems.


When Customer Feedback Leads, Positive Metrics Follow

It’s an inarguable fact: Metrics are useful. But they’re missing something—they don’t tell you the whole story. Quantitative metrics can tell you that a problem exists, but they can’t tell you what exactly the problem is or why it exists. Product managers are often tasked with increasing KPIs on their products, but they can’t do that by simply analyzing numeric data on its own.

To uncover the full story, it’s up to product managers to gather and interpret customer feedback and reveal what the metrics can’t—why the KPIs are changing (or not). In other words, you have to gather qualitative data.

Peel Back the Layers of Customer Feedback

Customer feedback has several layers. As I’ve learned over time in talking to customers, it’s often not about what people say, but rather what they don’t say. For example, when customers are struggling to accomplish their goals in your product, they will often come to you with specific solutions as how to solve the problem. It would be easy to take that one piece of feedback, add it to the roadmap and implement it directly as described. But it’s important to understand that people’s solutions are often based on their own personal preference or point of view, and that may not align with your entire customer base.

If you implement only one new tactic this year, make it this: Get out of the office and observe your customers using your product. That is the only way you will truly be able to figure out the true nature of your customers’ problems, why they exist, and how to solve them. Schedule an interview or user test with a new customer once a week (or at least every two to three weeks), and dig deep into the issues you discovered while reviewing your metrics.

It’s in these user tests that you are able to see problems occur in their natural environment. Your customers will show you their workarounds. They will tell you stories about how they “make do” using other products. You will observe their frustration as they try to do something your product doesn’t allow, or as they get confused about what to do next.

Proving That the Problem is Real

Since customers tend to start out by telling you the solutions they want, getting to the bottom of the problem is not always as straightforward as we’d like. Thankfully, there are three rules that can help you navigate the conversation to get you the answers you need.

1. Look to answer the four W’s.

In this case, the four W’s we are referring to are Who, What, Where and Why.

– Who has the problem? Which customer group are you really focused on? Most products have more than one persona. Thus, it’s important that you understand which persona you’re speaking to.

– What is the nature of the problem? You want to get to the point where you can explain the problem simply, preferably in one to two sentences. Additionally, you need to be able to communicate to your team how you know that this particular customer is experiencing this particular problem. So, ask your customer to walk through the product and make note of where they get stuck.

– Where does the problem arise, or in what context do your customers experience the problem? When you work to come up with a solution, you’ll want to make sure you do so in the areas your customers are most negatively affected.

– Why is the problem worth solving? Is this problem affecting a large portion of your customer base? And if so, what does the problem mean for your brand, revenue and trust in your product.

2. Avoid asking leading questions

Leading questions will get you the answers you want, but may not get you the answers you need. They are questions that tell the listener the answer you want them to give you.

Generally speaking, there are two ways to spot leading questions. The first indication is that the question can be answered with a yes or no. These closed questions seem pretty harmless, but they tend to be asked with a tone that puts emphasis on either the ‘yes’ or the ‘no’, thereby giving the listener the indication as to which you want them to say. The second indicator of leading questions is that they tend to have a solution in the question, e.g.“Would you use product X more if it had feature Y?”

3. Seek to disprove your assumptions

When building products, we make assumptions regarding what we believe to be true about our product and our customers. It’s appealing to try and confirm that we’re right, but when going out and talking to customers, you’ll actually get to the bottom of problems (whether or not they are real and worth solving) much more easily if you try to prove yourself wrong. Ultimately, this will help you avoid trying to solve the wrong, or less important, problems.

In your interviews, the first step is to avoid bias in your questions. Particularly Confirmation Bias, which happens when you (possibly unconsciously) interpret what your customer says and does in a way that confirms your preconceptions. This also shows up when you only ask questions based on what you expect to see. Additionally, you should watch out for Diagnosis Bias, which is when you form an initial impression and are then not open to other possibilities, either based on your initial assumptions or based on something that happens early on in the user test.

Secondly, ask questions to get an opposite answer than what you think to be true. So, if you believe a feature is good, ask your question in a way that will help you to determine how it could be bad. “If feature X fails, what are your workarounds?”

Back to the Beginning—Turning Feedback Into Metrics

As I said in the beginning, quantitative metrics can tell you that there is a problem, but they won’t tell you what that problem is or why it exists; qualitative data is how we get real understanding. But when it comes to measuring whether or not we truly solved the problem we set out to solve, it’s important that we bring it back to numbers. So as you decide on product changes based on what you learned,ensure that those changes are measurable by the KPIs you set out to change.

As your customers experience your solution, your metrics will adjust to reflect the impact of your changes. If you’ve made changes that truly solve the problem, you’ll see a positive impact in your metrics. If you missed the mark, as sometimes happens, your numbers will show that too. Either way, this step is really just taking you back to the beginning, as this is an ongoing feedback loop.

As product managers, we need to constantly evaluate our product and repeat this cycle over and over: Review metrics to discover where to focus our time, talk to customers to understand the true nature of their problems, and then measure whether or not our changes had a positive or negative impact (and by how much).

Metrics are the beginning and end of the story. They indicate to you that you need to change something, and they tell you when you can move your focus to something else. But in the middle, getting quality customer feedback is essential to making decisions that will move your metrics in the direction that will positively impact your customers and business. More importantly, those conversations lead to solid understanding of your customer. And understanding your customer and their needs is what leads to increased metrics all-around, no matter what conversion metric you measure.


So while metrics are incredibly important, don’t forget that you have humans using your product. In the words of Harvard Professor Youngme Moon, “If we pay attention to things that we can measure, we will only pay attention to the things that are easily measurable. And in the process we will miss a lot.” We miss the human experience.


So, what customer will you interview this week?


Article originally published in Mind the Product

No Joy From Facebook’s “Paper”


So I downloaded Paper by Facebook the other day (aside: why they just didn’t name it “Facebook” I have no idea). I watched the icon fill in with anticipation and got really excited when “Loading” changed to “Paper”. Of course a bit of wind went out of my sails when I opened the app and immediately got a tutorial about the app’s features. I did my best to quickly dismiss all of that and get to the app, so I could discover the awesomeness that was supposed to exist within my four inch screen, but they kept following me. Everywhere I went within the app, there were more directions! They didn’t seem to stop. At this point, I’m not even certain if they ever do.

I’ve had a saying since I began building apps which is “If I have to tell you how to use it, I didn’t do my job right.” I love user tests. I love them because when we build products we’ll get feedback (generally on the pain points) but we don’t know all of the experiences people have with our products at every moment. So user tests give us the opportunity to see and hear the reactions in the moment. And the tests that validate why I do what I do are the ones where I get to watch people’s faces light up when they discover something new. It may not be something “innovative” or completely novel, but the feature or action was there when they needed it and the app did what they thought it should do with each interaction.

LukeW has a post about Just In Time Education with mobile design that concludes with “The trick to getting just in time education right is revealing useful information when people actually need it not when they don’t.” And functionally, I think that is spot on. But the thing that I kept thinking when using Paper was that its not any fun. When we talk about UX  we often talk about it from the perspective of helping a user get from one place to the next and complete a task, but I firmly believe we also need to talk about the amount of joy someone feels when they are doing that task. After every action, it seemed like, Paper kept telling me more and more about their features. Why? I ask you, Facebook, is your app so complicated that I wouldn’t be able figure it out? And to make it even worse, these instructions were in the form of a popup. That I had to dismiss. So they stopped me from doing the action I was trying to do, to tell me either how to do it, or how to do something else. I really just wanted to scream at this app “What, do you think I am an idiot, Facebook? Do you think I don’t know how to use my phone? Or that I don’t understand the concepts of Facebook? You’ve been around for ten years now. I think I get it. I read a post. I can like it, comment, or share. You didn’t fundamentally change that. You just changed the packaging around it. If your new app is any good, I’ll figure it out!!!” But if I had yelled that at the app, I suppose the people around me would have thought me insane. So I didn’t. Instead I quietly backgrounded Paper and went to Twitter.

So I guess the point here is that its important that we think about ease of doing tasks when building products, but its equally important to think about how people feel while doing the tasks. Apathy does not create promoters. Joy creates promoters. And as builders of products, we need to think about that with every decision we make – How much joy will this decision give or take away?