Webinar Series Part 4 of 4: The Real MVP

Minimum Viable Product (MVP) is a term I hear often and, unfortunately, hear used incorrectly. Though, it’s understandable. The definition that Eric Ries popularized in his book, The Lean Startup, is a bit confusing. As a lean practitioner for almost ten years, I offer up a revised definition in hopes that product teams can start using MVPs to truly roll out Build-Measure-Learn loops in the ways The Lean Startup encourages us to do.

At my current company, Globant, I’ve created a curriculum and have been leading a webinar series to help product managers and others on the team focus on doing work that helps them solve customer problems and validate that they have solved them. In this installment, The Real MVP, we discuss an alternate definition of the term Minimum Viable Product, and articulate ways that teams can start executing MVPs to maximize learning.

Webinar Series Part 3 of 4: Metrics That Matter

Tracking metrics is easy. Add a Google Analytics snippet and you’ll get visits, time on page, and bounce rate within hours. The question is, what changes will you make to your product when you know that information? Making data-driven decisions is all the rage, and it matters what data you look at to help you make those decisions.

At my current company, Globant, I’ve created a curriculum and have been leading a webinar series to help product managers and others on the team focus on doing work that helps them solve customer problems and validate that they have solved them. In this installment, Metrics That Matter, we discuss the difference between vanity and actionable metrics and how to get to actionable metrics that help you move your product forward.

Webinar Series Part 2 of 4: Making the Most of User Interviews

In product management, we’re often told to talk to users. Doing so can be intimidating and scary if we’ve never done it before. The elusive “User Interview” can bring up a lot of emotion, both for the interviewer and the interviewee, and also for the stakeholders who are depending on the team to help ship a new product to market.

At my current company, Globant, I’ve created a curriculum and workshop series to help clients and internal team members learn a different way of approaching product development. To get the word out, we created a four-part webinar series to illustrate what you might learn in the workshops. The second one, Making the Most of User Interviews provides some actionable tips for folks in product development to get started talking to their users today.

Webinar Series Part 1 of 4: Hey, What’s Your Problem?

In product management, we often get presented with solutions:

Sally wants to take a picture.
John needs to call a cab.

Unfortunately, simply building these solutions means we sometimes miss relieving real pain for our users.

At my current company, Globant, I’ve created a curriculum and workshop series to help clients and internal team members learn a different way of approaching product development. To get the word out, we created a four-part webinar series to illustrate what you might learn in the workshops. The first one, Hey, What’s Your Problem? discusses ways to approach product development by focusing on user problems.


Eric Ries Needs a Better Editor (MVPs Explained)

Eric Ries’ The Lean Startup is a very successful book—and an even more successful movement—amongst product developers. Even if those in product development haven’t read the book, they talk about concepts introduced and popularized in it, such as the ‘Build-Measure-Learn’ cycle and Minimum Viable Products (MVPs).

When defining the ‘Build-Measure-Learn’ cycle, Ries explains that “the fundamental activity of a startup is to turn ideas into products, measure how customers respond, and then learn whether to pivot or persevere.” An MVP, he says, is “that version of the product that enables a full turn of the Build-Measure-Learn loop with a minimum amount of effort and the least amount of development time.”

Whenever I work with a new team, I inevitably hear the phrase “let’s just build an MVP”. Teams resort to this strategy when they are given work that either has a lot of unknowns, doesn’t clearly solve a user problem, or doesn’t align with the goals of their company. What these teams often mean, though, is “let’s build whatever we can build in the time frame that someone has set for us (or that we have set for ourselves) that gets us closer to our grand vision.” The MVP often ends up being a small Version One.

Others in product development have noticed this too, and have proposed a variety of alternatives to ‘MVP’ as a way to capture the nuance of Ries’ definition of a product. Minimum Valuable Product, Minimum Viable Experiment, and Riskiest Assumption Tests are all alternatives that have been proposed in the last couple of years. In his post Death of the Minimum Viable Product!, Steve Cohn, CEO of Validately, shows that the use of the word ‘product’ caused confusion amongst product people because they interpreted it as a ‘releasable product’. His analysis is spot on.

If you look past Ries’ definition of an MVP in The Lean Startup, and instead focus on the examples he uses, it becomes clear that his definition of MVP means something that you can use to run an experiment. Ries gives examples of an MVP, my favorite of which involves Food on the Table. Food on the Table’s product helps users plan meals by comparing their recipes to items on sale at their local grocery stores. The founders of Food on the Table experimented with solutions with customers for months without ever building software. They compared local grocery store sales and meal preferences for each individual customer, printed out recipes and meal plans, and, at first, went shopping with each customer until they had so many customers that the team had to build software to keep up with demand.

It’s fair to say that the printed-out paper the founders provided was, in fact, a product. The problem is that software—or even hardware—developers don’t tend to create and keep version numbers when they move from paper or physical prototypes to software. Calling the printed-out paper that the founders provided to their customers in the beginning a ‘version’ of their product is a lovely idea. But when Ries wrote the definition of MVP, he didn’t mean for ‘version’ to span the entire development process from “I have an idea” to a marketable, sellable product. Because of this, the use of the word ‘version’ didn’t fit with the definition of the word that readers typically understand.

How, then, can we rewrite the definition to capture the nuance that Ries illustrates through examples?

Since an MVP is something that can be used to test out a hypothesis, we could start by saying that rather than an MVP being a “version of the product”, it’s a “testable solution to a problem.” Understanding an MVP in this way frees us up to imagine all of the potential mediums that a solution might be, such as paper, digital, or physical representations.

Let’s move on to the next part of Ries’ definition: “that enables a full turn of the Build-Measure-Learn loop”. Building could, again, involve any medium. So let’s focus on ‘Measure’. To measure something in a meaningful way, a measurement needs to follow the SMART metric methodology: Specific, Measurable, Actionable, Realistic and Time-Bound. Whatever is being tested must have an objective right and wrong answer in a defined period of time. That way, it’s possible to make a decision easily about whether to continue with or change the course of your product. If a test doesn’t have a metric that is SMART, it’s too easy for teams to not learn something from the test and instead succumb to the sunk-cost fallacy, in which they continue to build simply because they have put time and effort into the product. I’d therefore propose that this section of the definition be changed to “used in an experiment that can generate an objective right or wrong answer in a given amount of time”.

The end of the definition, “with a minimum amount of effort and the least amount of development time”, follows standard lean practices. There’s nothing inherently wrong with or confusing about the sentence. But for the clarity of the entire definition, let’s shorten it to “with the least amount of time and development waste.”

When we pull it all together, we arrive at a definition of MVP that is much clearer and captures the nuance of what Ries was trying to teach: “The MVP is a testable solution to a problem used in an experiment that can generate an objective right or wrong answer in a given amount of time with the least amount of time and development waste.”

By being more specific in the definition, I’m hopeful that teams will not resort to saying “let’s just build an MVP” when they are caught in situation in which they are asked to build something without understanding which problem they are solving. Rather, they will help their stakeholders and teammates discover user problems, and then use their MVPs to validate if they are indeed worth solving.


Originally written for the blog on L4 Digital.com

When Customer Feedback Leads, Positive Metrics Follow

It’s an inarguable fact: Metrics are useful. But they’re missing something—they don’t tell you the whole story. Quantitative metrics can tell you that a problem exists, but they can’t tell you what exactly the problem is or why it exists. Product managers are often tasked with increasing KPIs on their products, but they can’t do that by simply analyzing numeric data on its own.

To uncover the full story, it’s up to product managers to gather and interpret customer feedback and reveal what the metrics can’t—why the KPIs are changing (or not). In other words, you have to gather qualitative data.

Peel Back the Layers of Customer Feedback

Customer feedback has several layers. As I’ve learned over time in talking to customers, it’s often not about what people say, but rather what they don’t say. For example, when customers are struggling to accomplish their goals in your product, they will often come to you with specific solutions as how to solve the problem. It would be easy to take that one piece of feedback, add it to the roadmap and implement it directly as described. But it’s important to understand that people’s solutions are often based on their own personal preference or point of view, and that may not align with your entire customer base.

If you implement only one new tactic this year, make it this: Get out of the office and observe your customers using your product. That is the only way you will truly be able to figure out the true nature of your customers’ problems, why they exist, and how to solve them. Schedule an interview or user test with a new customer once a week (or at least every two to three weeks), and dig deep into the issues you discovered while reviewing your metrics.

It’s in these user tests that you are able to see problems occur in their natural environment. Your customers will show you their workarounds. They will tell you stories about how they “make do” using other products. You will observe their frustration as they try to do something your product doesn’t allow, or as they get confused about what to do next.

Proving That the Problem is Real

Since customers tend to start out by telling you the solutions they want, getting to the bottom of the problem is not always as straightforward as we’d like. Thankfully, there are three rules that can help you navigate the conversation to get you the answers you need.

1. Look to answer the four W’s.

In this case, the four W’s we are referring to are Who, What, Where and Why.

– Who has the problem? Which customer group are you really focused on? Most products have more than one persona. Thus, it’s important that you understand which persona you’re speaking to.

– What is the nature of the problem? You want to get to the point where you can explain the problem simply, preferably in one to two sentences. Additionally, you need to be able to communicate to your team how you know that this particular customer is experiencing this particular problem. So, ask your customer to walk through the product and make note of where they get stuck.

– Where does the problem arise, or in what context do your customers experience the problem? When you work to come up with a solution, you’ll want to make sure you do so in the areas your customers are most negatively affected.

– Why is the problem worth solving? Is this problem affecting a large portion of your customer base? And if so, what does the problem mean for your brand, revenue and trust in your product.

2. Avoid asking leading questions

Leading questions will get you the answers you want, but may not get you the answers you need. They are questions that tell the listener the answer you want them to give you.

Generally speaking, there are two ways to spot leading questions. The first indication is that the question can be answered with a yes or no. These closed questions seem pretty harmless, but they tend to be asked with a tone that puts emphasis on either the ‘yes’ or the ‘no’, thereby giving the listener the indication as to which you want them to say. The second indicator of leading questions is that they tend to have a solution in the question, e.g.“Would you use product X more if it had feature Y?”

3. Seek to disprove your assumptions

When building products, we make assumptions regarding what we believe to be true about our product and our customers. It’s appealing to try and confirm that we’re right, but when going out and talking to customers, you’ll actually get to the bottom of problems (whether or not they are real and worth solving) much more easily if you try to prove yourself wrong. Ultimately, this will help you avoid trying to solve the wrong, or less important, problems.

In your interviews, the first step is to avoid bias in your questions. Particularly Confirmation Bias, which happens when you (possibly unconsciously) interpret what your customer says and does in a way that confirms your preconceptions. This also shows up when you only ask questions based on what you expect to see. Additionally, you should watch out for Diagnosis Bias, which is when you form an initial impression and are then not open to other possibilities, either based on your initial assumptions or based on something that happens early on in the user test.

Secondly, ask questions to get an opposite answer than what you think to be true. So, if you believe a feature is good, ask your question in a way that will help you to determine how it could be bad. “If feature X fails, what are your workarounds?”

Back to the Beginning—Turning Feedback Into Metrics

As I said in the beginning, quantitative metrics can tell you that there is a problem, but they won’t tell you what that problem is or why it exists; qualitative data is how we get real understanding. But when it comes to measuring whether or not we truly solved the problem we set out to solve, it’s important that we bring it back to numbers. So as you decide on product changes based on what you learned,ensure that those changes are measurable by the KPIs you set out to change.

As your customers experience your solution, your metrics will adjust to reflect the impact of your changes. If you’ve made changes that truly solve the problem, you’ll see a positive impact in your metrics. If you missed the mark, as sometimes happens, your numbers will show that too. Either way, this step is really just taking you back to the beginning, as this is an ongoing feedback loop.

As product managers, we need to constantly evaluate our product and repeat this cycle over and over: Review metrics to discover where to focus our time, talk to customers to understand the true nature of their problems, and then measure whether or not our changes had a positive or negative impact (and by how much).

Metrics are the beginning and end of the story. They indicate to you that you need to change something, and they tell you when you can move your focus to something else. But in the middle, getting quality customer feedback is essential to making decisions that will move your metrics in the direction that will positively impact your customers and business. More importantly, those conversations lead to solid understanding of your customer. And understanding your customer and their needs is what leads to increased metrics all-around, no matter what conversion metric you measure.


So while metrics are incredibly important, don’t forget that you have humans using your product. In the words of Harvard Professor Youngme Moon, “If we pay attention to things that we can measure, we will only pay attention to the things that are easily measurable. And in the process we will miss a lot.” We miss the human experience.


So, what customer will you interview this week?


Article originally published in Mind the Product

9 Ways I Keep My Space Simplified

I was chatting with a co-worker the other day about my purging methods and ways to keep the quantity of things I own to a minimum and he suggested that I write down my tips. So, here are the rules I live by:

1. If I haven’t used it in a year, I don’t need it.

2. “Memories” are the exception to rule number one. But, I confine myself to one memory box. And its small enough to fit under my bed. If you were going to try this, I’d suggest just giving yourself the rule of having one box that you are able to move by yourself, however big that box may be.

3. Start purging clothes first.
At the end of summer I always go through my summer clothes and add the items to the donate pile that I had saved from the previous summer but didn’t at all use that summer.
If I buy a new version of something that is better than the old version, I get rid of the old version.
– Because I am not a painter, landscaper, mechanic, etc, I don’t need 12 pairs of “old jeans”. One is probably enough.
– Unless I were to be actually in the process of losing weight, meaning I’m actively working out, dieting, giving birth, having a massive tumor removed, etc, I resign myself to the fact that I will not wear my skinny clothes again. By the time I were to make the decision or effort to actually do the noted items above, fashions would change and I’d probably want the joy of going out and buying new clothes anyway.

4. Buy a few high quality things instead of a lot of cheap things. This particularly applies to furniture. Oh, and a bonus tip – I always buy furniture that allows me to store stuff in it. Open shelving creates an environment for more clutter.

5. I don’t buy the extra items in the “Buy 3 get 1 free” deals. If in the next month, I am only going to use 1, then I just buy one. If I was going to use all 4 (or even 3) in the next month, or before I can possibly get to the store that sells that thing again, then by all means, I’ll buy all 4 (Cereal comes to mind here. I do love my cereal). Why? See the following reasons:
The time value of money means something. According to whomever wrote this wikipedia article, the time value of money is “the principle that a certain currency amount of money today has a different buying power (value) than the same currency amount of money in the future. The value of money at a future point of time would take account of interest earned or inflation accrued over a given period of time.” So basically, if I spend $1 today on a bar of soap I don’t use, I’ve actually lost money because I didn’t use nor need the soap, and that $1 might have the buying power of $1.03 in the future.
My acquisition costs of new items are generally fairly low. I visit Target pretty regularly. I get perishable items there and other items that need routine replenishing. Because of that, if I need that additional bar of soap, the acquisition cost to pick that item up when I am getting others is nonexistent. If acquisition costs were high, I feel there is value in buying extra.
The cost to store the additional items has a negative effect on my life (or diminishing marginal utility) as I need to find a place to store and organize the additional items, often then resulting in the need to buy additional organizational items or furniture.

6. Get the items you are purging out of the house as fast as possible. So if I go through my closet and bag up clothes, those bags go by the door to take out when I go out next or I immediately put them in my car. The longer those bags sit there, the higher probability that I will get comfortable with them sitting there.

7. I don’t beat myself up when I can’t let go of something right away. Sometimes, it takes evaluating something a week later or a month later to finally be able to get rid of it. Example: when I just moved I brought along some scarves that I thought I wanted. Then when I was unboxing them and putting things away in my closet, I realized, I never wear those scarves and had others to take their place. So I got rid of them then.

8. I absolutely, positively, never take tchotchke. If I go to an event and they are giving out keychains, I tell them to keep it. Water bottles, t-shirts, notebooks, coffee cups, etc. I probably won’t like their water bottle, I have enough t-shirts, who writes things in notebooks? and my coffee cup set matches and is phenomenal so I don’t need to damage its beauty with some weird cup that advertises a brand I probably don’t care about. If I take it, it’ll probably just end up in a landfill, so I let them keep it and give it to someone who might actually want it.

9. I don’t participate in the gift-giving aspect of holidays. Its much better for me to just buy my own stuff and others to buy their own stuff than for us to try and guess what each other likes within the dollar range we can spend on them. Here are two episodes of my favorite podcast, Planet Money, that helps to explain this – Making Christmas More Joyful, And More Efficient and The Most Wasteful Time Of Year.

Learning to purge takes time. Clothes are always the easiest. But after awhile it all just becomes habit. And I find myself only buying things I really like. So I’m really happy with my physical space and enjoy being in my place.


My asterisk to this is, yes, absolutely, your situation is different and there are always exceptions to every rule. These principles have worked for me and may not work for you. I am not a tax advisor. I do not get paid to give financial advice. Your grandmother might be mad if you get rid of that ugly gravy boat she gave you for your wedding. And no, this is not an offer to clean your house.

No Joy From Facebook’s “Paper”


So I downloaded Paper by Facebook the other day (aside: why they just didn’t name it “Facebook” I have no idea). I watched the icon fill in with anticipation and got really excited when “Loading” changed to “Paper”. Of course a bit of wind went out of my sails when I opened the app and immediately got a tutorial about the app’s features. I did my best to quickly dismiss all of that and get to the app, so I could discover the awesomeness that was supposed to exist within my four inch screen, but they kept following me. Everywhere I went within the app, there were more directions! They didn’t seem to stop. At this point, I’m not even certain if they ever do.

I’ve had a saying since I began building apps which is “If I have to tell you how to use it, I didn’t do my job right.” I love user tests. I love them because when we build products we’ll get feedback (generally on the pain points) but we don’t know all of the experiences people have with our products at every moment. So user tests give us the opportunity to see and hear the reactions in the moment. And the tests that validate why I do what I do are the ones where I get to watch people’s faces light up when they discover something new. It may not be something “innovative” or completely novel, but the feature or action was there when they needed it and the app did what they thought it should do with each interaction.

LukeW has a post about Just In Time Education with mobile design that concludes with “The trick to getting just in time education right is revealing useful information when people actually need it not when they don’t.” And functionally, I think that is spot on. But the thing that I kept thinking when using Paper was that its not any fun. When we talk about UX  we often talk about it from the perspective of helping a user get from one place to the next and complete a task, but I firmly believe we also need to talk about the amount of joy someone feels when they are doing that task. After every action, it seemed like, Paper kept telling me more and more about their features. Why? I ask you, Facebook, is your app so complicated that I wouldn’t be able figure it out? And to make it even worse, these instructions were in the form of a popup. That I had to dismiss. So they stopped me from doing the action I was trying to do, to tell me either how to do it, or how to do something else. I really just wanted to scream at this app “What, do you think I am an idiot, Facebook? Do you think I don’t know how to use my phone? Or that I don’t understand the concepts of Facebook? You’ve been around for ten years now. I think I get it. I read a post. I can like it, comment, or share. You didn’t fundamentally change that. You just changed the packaging around it. If your new app is any good, I’ll figure it out!!!” But if I had yelled that at the app, I suppose the people around me would have thought me insane. So I didn’t. Instead I quietly backgrounded Paper and went to Twitter.

So I guess the point here is that its important that we think about ease of doing tasks when building products, but its equally important to think about how people feel while doing the tasks. Apathy does not create promoters. Joy creates promoters. And as builders of products, we need to think about that with every decision we make – How much joy will this decision give or take away?