How to collect feedback you can actually use

Show your app to your cousin or your old classmates and the answer will be “It's nice, well done.” They mean it kindly. It tells you nothing. Rob Fitzpatrick wrote a whole book, The Mom Test, about this problem: almost everyone will lie to be polite if you ask whether your idea is good. His three rules are simple. Talk about their life, not your idea. Ask about specific things that already happened. Talk less and listen more.²

You can't build anything from praise. So when someone says “I'd definitely use this,” ask them a question about facts. What did they do the last time this problem came up?

Questions about the future get guesses. “Would you pay for this?” “Would you use a dark mode?” People answer in good faith and then don't do it. Eric Migicovsky built the Pebble smartwatch, and in a Y Combinator Startup School talk he told founders to ask about real events instead. His five questions are a good start. What's the hardest part of doing this job? Tell me about the last time you hit this problem. Why was it hard? What have you tried so far? What don't you like about those options?¹

For a project that already works, change the subject from the problem to your app. Ask them to talk you through the last time they opened it, what they were trying to do, and where they got stuck. If they stopped using it, ask what they did instead. That last answer often tells you more than anything else, because it names what you're really up against. Sometimes it's another app. Often it's a WhatsApp group, a notebook or an aunt who just handles it.

Jakob Nielsen of Nielsen Norman Group put the first rule of usability bluntly: watch what people actually do, and don't believe what they say they do. In his research, what people said they preferred was a weak guide to how well they did with a design.³

So hand your phone to someone, give them a task like “send ₦2,000 to this person” or “book a slot for Saturday,” and stay quiet. Don't explain. Don't point. Every time you want to say “just tap there,” write it down instead, because that's a problem you've found.

You don't need many people. Nielsen's research found that testing with five users shows you about 85% of the usability problems, and that you learn more from three small rounds than from one big one.⁴ Fix what the first five found, then test again with five new people.

Timing changes the answers. Ask straight after someone finishes a task, while it's fresh, and ask about that one task. A form that pops up the first time someone opens the app gets either nothing or “good app,” and a long survey sent a month later gets very few replies.

Once you have people who use it every week, a short regular check helps. Migicovsky liked a weekly survey based on the email app Superhuman. It asks one thing: how would you feel if you could no longer use this product? When around 40% pick “very disappointed,” he counts that as a strong signal that people need it.¹

Keep calls short, ten or fifteen minutes.¹ Respect people's time and their data. A voice note is fine if that's what they prefer, but ask before you record anything.

Feedback scattered across WhatsApp chats, DMs and your memory is close to useless. Keep one place for it: a simple sheet works. Give each one its own row: who said it, when, the exact words, and what they were in the middle of doing.

Once a week, read through the sheet and group the notes. You'll usually end up with four or five piles. Bugs. Screens that confused people. Features they missed. Grumbles about the price. Count each pile. If eight people couldn't find the save button and one person wants a dark mode, the save button wins, however loudly the dark mode person asks.

Weigh who said it as well. The woman who opens your app every morning and asks for one change gets more weight than a stranger who tried it once and sent ten complaints.

Not all feedback deserves an answer in the product. Leave these alone, or at least park them. Compliments, because they don't tell you what to change. Promises about the future, like “I'd pay for this if it had X,” unless they pay now. Requests from people who aren't the users you're building for. Feature ideas that would turn your app into a different app.

Treat a feature request as a clue. Someone wants an export button? Find out what they planned to do with the export. Maybe they only needed to send one receipt to their boss, and a share link does that in an afternoon. Migicovsky's advice was to test whether a feature matters before building it, for example by asking people to pay for it first.¹

When you fix something someone reported, tell them. Something like “You said sign-up kept failing on your Tecno. Fixed now, thanks for telling me” is plenty. People who see you act on what they said usually send more, and a few start telling friends about your project.

On Prow: Posting your project on Prow is free. When you post, there's a box for what you want feedback on. Whatever you type there shows on your project page, so people who try it know what to look at. Their feedback comes to your dashboard, one list for everything, and you can answer each person from there. On the Explore page people can sort by most feedback, so pages with plenty of it are easy to find.

• You ask about things that already happened, not what people would do.
• You've watched at least five people use it without helping them.
• You ask right after a task, not when someone first opens the app.
• Every piece of feedback goes in one sheet with the person's exact words.
• You count the piles each week and fix the biggest one first.
• You tell people when something they reported is fixed.