No innovator fails because they couldn’t build their product. They fail because no one found value in what they built.
Here’s what usually happens when someone gets a new idea:
Sadly, this is the typical cycle of innovation. Most new initiatives fail.
But successful innovators know that they have a very powerful tool at their disposal to significantly decrease the risk of innovation: experimentation.
Experiments are small bets that you make to see if what you believe to be true is actually true - to see if your predictions about the customer and the market are right. It’s something small that you do today to prove that you are spending time and money on the right things . . . on building something that people will buy.
In All in Startup, the reader meets Owen after he has followed the very process outlined above and wasted hundreds of thousands of dollars and a year of his life building something that wasn’t meeting any of his projections. What could he have done before he committed his available resources to executing his plan, to make sure customers would be waiting for him when the product was ready? He should have run some experiments!
But the hardest thing about experiments is running them correctly.
Here are some guidelines about what a good experiment looks like.
To run a good experiment, you need to determine and document these five elements before you begin:
What is it that you’re trying to learn? Or what are you trying to prove? What are the riskiest assumptions you’ve made about your idea? For Owen, his riskiest assumption was that people would buy half-priced, used bicycles online.
Your high school science classes taught you what you need to know for this part. This is a statement that you’re trying to prove true or false through the experiment. The result will be a “yes” or a “no” so you need to phrase your hypothesis appropriately.
For example, a good hypothesis would be based on this setup: “If I do this action, then this outcome will happen.”
The key is to make sure that your Hypothesis helps you get closer to the Goal you outlined above and reduce the risk of your riskiest assumption. The hypothesis can help you test whether you have identified the right customer segment, whether your target customers actually have the pain point you think they do, whether they perceive enough value in your solution to buy it, whether your solution actually solves their pain point, whether you’ve identified the right channels to target your customers, whether your supplier cost estimates are accurate, whether you’ve chosen the right price point for your product, etc.
The biggest mistakes in putting together a hypothesis include 1) Creating a hypothesis that isn’t measurable and lacks a clear beginning and end (ex. “People want to eat healthier”); 2) Creating a hypothesis that isn’t about a specific group of customers (ex. Everyone wants a car that gets at least 30mpg); 3) Creating a hypothesis that doesn’t help you reach your goal (ex. “If I send this survey to 100 people, 10% will fill it out”); or 4) Creating a hypothesis that isn’t refutable, meaning that it’s difficult or impossible to prove it false (ex. “Restaurants want more customers”).
Some example hypotheses Owen could have created:
“If I put up a landing page to sell half-priced, used bicycles, then 5% or more of the website visitors will preorder the bikes.” – Testing value
“If I set up a booth at a bike race for one day, I will sell at least 10 bicycles” – Testing customer segment
“If I spend $500 on facebook ads targeting people with cycling listed as an interest, at least 5 people will click on the buy button and enter their payment information” – Testing customer segment and marketing channel
“If I call 20 bike shop owners, at least 3 will come to my shop for a one-hour meeting to find out more about my bikes and whether they want to sell them in their store” – Testing distribution channel.
Who are you targeting with the experiment? How are you filtering who will participate and who won't? For instance, if Owen puts up a landing page to see if people interested in road bikes will want to buy his bikes, what kind of information is he gathering on the landing page to make sure that the right people are seeing his messaging before he decides whether it's working or not?
Tip: If you are having trouble limiting your target subject for the experiment, try first listing out people who wouldn't fit into your target subject. I.e., for Owen it's people who want to buy a $100 bicycle at Walmart or Target or perhaps people who are interested in roadbikes but have never actually purchased one because they think they are too expensive.
How are you going to conduct your experiment? What’s the time period? How do we know when the experiment has started and has finished? How many people will you target? Who will carry out the experiment?
A key question to ask yourself here is: is this the least amount of time and effort I can spend to test this hypothesis? Remember, this is supposed to be a small bet you can afford to lose. Too many people think their experiment is building a lighter version of the final product – taking 6 months to put together.
You should be able to run your experiment in under 2 weeks. I will frequently push my innovators to come up with a hypothesis and figure out a way to start the experiment within 24 hours.
Something of value that the subjects of your experiment have to give you in order to prove whether your hypothesis is true.
The key is that it be something painful for them to give up in order to demonstrate their sincerity. This can include anything from money to time to a commitment of certain resources. Basically anything that demonstrates customers will be willing to part with something they value in order to obtain the product or service you’re offering. You want to make sure that they aren’t just trying to be nice to you or lying about their intent for some other reason.
When you are trying to de-risk an idea, the worst kind of evidence you could gather is a false positive (i.e., perceived interest from people who don’t actually see value in your product) because it gets you excited about moving forward and going All In on the idea. Simulate a world where your product already exists, and see if your customers will give you the currency you think you deserve.
In the Owen hypothesis examples above, Owen is asking for specific actions, a commitment of money or commitment of time to demonstrate whether he’s identified the right customer segments, marketing channels and distribution channels.
Before you begin your experiment, it’s important to define what success and failure will look like. If success is having 25 percent of customers give you currency, what does it mean when only 15 percent provide it? Has your experiment failed?
You need to set up these parameters before you begin the experiment, so that you’ll objectively understand the outcome and not be forced to debate what the results mean with your team.
Additionally, a friend of mine, Justin Wilcox, suggests that innovators write out two separate plans of action to pursue depending on whether the experiment reveals the hypothesis to be correct or incorrect. He emphasizes that this should be done before conducting the experiment. Some people have such a hard time deciding what to do if an experiment doesn’t go as they had planned that they end up making up a justification of why it was a success and allowing themselves to keep moving forward on their idea.
OK. Those are the elements you need to determine and document before you begin your experiment. And while this experiment framework can seem cumbersome, remember, it is your single greatest asset in reducing the risk that goes with creating something new.
I’ve been thinking about my first motorcycle accident. I’m not counting the time when I drove my Kawasaki Ninja to my friend Eric’s house and, while showing it off, fell over and needed his help to pick it back up. No, this was a lot more serious
I was attempting to make a left turn at a light and I think I must have slipped in something greasy on the ground, lost control and ended up running the bike into the curb. As I hit the curb, I flew off the bike and landed on my head and shoulder. My head was ok, but I really hurt my rotator cuff in my right shoulder. It’s been acting up a lot lately, so I’ve been reminiscing over the accident.
Riding a motorcycle is really dangerous. And some of the navigation is extremely counter intuitive. For instance, if you want to make a left turn while traveling above 5mph, you need to push the handlebars in the opposite direction (to the right) in order to turn left. It feels really weird when you first do it. And it takes a lot of practice to get used to it. I can’t really explain to you why it works, but I know that it’s counter to everything your gut instincts tell you.
We’ve all heard the story about the boiling frog. That if you put a frog into a pot of boiling water, it will instantly jump out, but if you set the frog in a pot of cold water and slowly turn up the heat, it will gradually cook to death. (For the record, this is one experiment I’ve never conducted. No frogs were harmed in the writing of this post.) The frog story is how I think about motorcycle riding and entrepreneurship.
Motorcycles are obviously dangerous. They're like the pot of boiling water. But entrepreneurship can be just as detrimental to your health as riding motorcycles; it’s just a lot more subtle. I’ve seen people lose their life savings, their relationships, and their health struggling to “make it” as an entrepreneur. To some extent, I’ve experienced each of these stresses myself through my entrepreneurial journey. It’s just hard to see when you are in a pot where the water is getting hotter and hotter, where the amount of risk you are getting yourself into is growing and growing.
And as is the case with riding a motorcycle, there are several counterintuitive principles you have to master to succeed as an entrepreneur. Just like pushing the handlebars in the opposite direction of the way you want to turn, there are things you need to do to be successful as an entrepreneur that feel totally unnatural, counter to your instincts, and downright hard to explain.
The most important of these counterintuitive principles is that people buy solutions to their problems, not products and services.
The more serious the pain you can solve, the more it means you can charge for your product. You can bank better margins, enjoy much shorter sale cycles, and spend a lot less on sales and marketing because your customers will be evangelizing the product on your behalf through powerful word of mouth testimonials.
Sounds pretty good, right? But unfortunately, 95% of entrepreneurs don’t follow this simple rule. That number is probably even higher than 95%, but I want you to keep reading and not get hung up on it.
It’s not that most entrepreneurs deny that solving problems is important. It’s that they go out of business because they build solutions for problems that don’t actually exist.
Here’s what usually happens:
An entrepreneur will get an idea and start thinking of all the possibilities of what it could turn into, the impact it could have on the world, and all the money it could generate.
Next, they will build the product. They spends a lot of time and money trying to build the most comprehensive version of it, rarely showing it to anyone because they want it to be perfect before potential customers see it. First impressions are everything!
Then, they brand the idea. They spend time and money developing a catchy name and a logo, purchasing a domain, building a website, creating marketing materials, etc. This has to look professional!
Finally, they go out looking for customers and, more often than not, strike out big time, causing them to realize that something is wrong with the initial idea. They revisit the idea and start brainstorming how to make it better. And then they repeat step one through four all over again, spending a lot of time and money, without making any forward progress.
This is the startup loop of despair. It can last anywhere from a few months to a few years, and it usually ends when the entrepreneur finally runs out of money. When they are fully cooked!
But successful entrepreneurs know that the startup loop of despair is completely avoidable. They know that once you come up with a great idea, the very next step should be to find potential customers and determine if your product is even worth building.
There is only one way to diagnose whether your idea solves a real problem. You have to conduct a Problem/Solution Fit Test.
A lot has been written about this process, but many still have trouble applying it because it’s so counterintuitive. It feels weird and awkward and it’s hard to know when you are doing it right. So I wanted to break it down and try to do my best to explain it.
What exactly are we testing?
1. The Problem – Does a specific group of customers have a migraine problem?
2. The Solution – Does my solution solve the problem?
How are we testing these two things?
We actually use different tools for each part of the test.
1. Diagnosing migraine problems is most effectively done through customer interviews. Believe it or not, customer interviews are extremely tricky because both you and the customers are coming into the conversation with a number of biases, so we have to conduct them in a very specific way to make sure we are getting honest feedback that we can use.
2. Solution testing is best done through objective experimentation. Humans are terrible at predicting the future. So we can’t really interview our customers to understand if the solution will work. We have to find a way to simulate the future and objectively test how customers will actually behave.
Conducting the Problem/Solution Fit Test before building your product will guarantee that you will build a product people actually want by figuring out which features and benefits are the most valuable. Above all, this means your startup will actually generate revenue.
But what about Henry Ford and Steve Jobs?
I get asked this question a lot. You’ve heard the quote, Henry Ford once said that if he were to ask people what they wanted, they would have said “faster horses.”
He’s totally right. You can’t ask people what they want. They are terrible at predicting the future and sometimes they plain out lie. But you can ask them about the past or the present. About current problems they are facing. And if Henry asked potential customers if they had pain about trying to find transportation to their jobs or to visit friends or family, they would have talked his ear off about all the different things they’ve tried to solve the problem.
Ok, and what about Steve Jobs? Most people think he never tested any of his products. He must have predicted what we would want!
Yes, Steve Jobs is a genius. There is no doubt about that. He beautifully combined art and technology, but not all his guesses about what the people wanted were right. The original Macintosh that Jobs created was a failure. It didn’t have a fan because Jobs thought it distracted from the calm of the computer and ended up frying all of the internal parts. The Lisa computer he created was also a failure. His NEXT company, created another personal computer that was a huge flop. But he had something like a hundred million dollars from his initial success and the ability to make lots of big bets without going broke.
If you don’t have tens of millions of dollars you can afford to lose just trying out different startup ideas, then you’re not Steve Jobs.
But here’s some good news. You can use the Product/Market Fit Test to significantly de-risk your bets before you go all in. The best way to de-risk an idea is to make sure people want it before you spend too much time and money creating it. If you aren’t solving a migraine problem, you’ll need to spend a lot of time and money educating people about your existence and and convincing them that they can benefit from your product. It’s very expensive to convince people that they need or want something that they really don’t.
When All In Startup first came out, I put together a list of customer interview rules to help entrepreneurs conduct the Problem/Solution Fit Test. You can find the original list here. Since then, I’ve received hundreds of follow up questions. So I’ve decided to spend the next few posts diving deep into the process of interacting with your customers.
Whether you are an entrepreneur with a crazy idea or a large company with thousands of existing clients, the upcoming articles on interacting with your customers will help you significantly de-risk your business.
You will learn:
How to get honest feedback from your customers
Which questions are the most valuable to ask
How to conduct small experiments to make sure customers are interested in your product before you invest too much time and money creating it
To make sure you receive these posts when they become available, subscribe here. And please either send me an email or add a comment to this post to let me know specific questions you have about the process to make sure I answer them.
I have a small confession to make. As I thought about the bike accident over the last 15 years, I’ve always said there must have been some grease in the road to slip me up. Some kind of discharge from a car that made the road hazardous. But as I was writing this post and thinking about my experience level at the time, I think that it’s much more likely that I hesitated. That I went with my gut and what my instincts were telling me to do instead of following the framework that I knew would be effective.
I know how scary it is to start something new. I know what it feels like to put everything on the line. I’ve felt the pain of what happens when your instincts take over. My objective is to help both you and me find a way to follow the counterintuitive rules of the road to get us all home safe.
Thanks for reading.
A few weeks ago, at the National Association of Community College Entrepreneurship (NACCE) Conference, I took part in a panel discussion on the subject of preparing our future workforce for entrepreneurial thinking.
As companies increasingly express an interest in creative, innovative employees, one of the hottest questions in entrepreneurship education has become, “How do we prepare graduates to solve real problems and find opportunities for their employer rather than just show up to a job?”
The first question I received on the panel was “how are we doing as a community of entrepreneurship educators in preparing this future workforce?”
I was quiet for a few seconds. Frankly, I was nervous about what would happen if I said the first thing that came to my mind. I prepared myself to get booed off the stage and said, “I think we’re doing a terrible job. In fact, I think that we are actually killing innovative and entrepreneurial thinking through the classes that we teach.”
They didn’t erupt in boos. They let me continue.
1. Students are pitching to professors, not customers.
In many entrepreneurship classrooms, professors make judgment calls about good ideas and bad ideas. Sometimes students pitch their ideas in business plan competitions to panels of experts that decide the value of a potential company. In the real world, logic and expertise can’t predict customer behavior. Even the top professional startup investors are wrong about predicting which startups will be successful 90% of the time. The only way to create a successful company is to discover what customers actually want through direct interactions, not assumptions. Instead of making judgments, we should push students to interact with potential customers and conduct experiments to see if they can simulate sales. If a student is required to appease an internal source of authority, like a professor, for the sake of their grade, then they won’t learn to respect the true source of authority, the customer.
2. Classroom work isn’t giving students butterflies in their stomachs.
Creating a successful company is not as simple as checking things off a to-do list. Yet many professors still give students a list of tasks—create a business plan, interview an entrepreneur, read a book—as if there is an easy roadmap to building a million dollar company. Entrepreneurship is an emotional roller coaster. It’s scary and exciting all at the same time, and, above all else, it’s fraught with uncertainty. If teachers aren’t giving students butterflies in their stomachs – making them feel overwhelmed or uncomfortable – then they aren’t preparing students for the challenges of entrepreneurship. And they certainly aren’t teaching them to develop entrepreneurial thinking. Instead, teachers who don’t push students out of their comfort zone are simply reinforcing traditional 9 to 5 employee thinking. There’s nothing wrong with a 9 to 5 employee, but it’s not the mindset companies are currently looking for as they interview business school graduates.
If you want to give your students real butterflies, then give them an objective and ask them to figure out how to achieve it. Better yet, send them out on some real experiential activities, like the activities suggested in this curriculum. There is no doubt that uncertainty in a classroom makes students uncomfortable. They like having tasks that make it easy for them to walk the road to success. But, if we keep teaching students entrepreneurship that only works in a classroom, and not in the real world, we aren’t doing them any favors. In fact, we’re setting them up to fail themselves and their employer.
It is our responsibility to push students to fight through difficulty and uncertainty. If your students are uncomfortable, that’s a good thing. When they enter into the workforce, they will be armed with an entrepreneurial mindset, arguably the most important key to success in business over the next 20 years. The great thing about the attendees of the NACCE conference is that most of them already know these two potential barriers to the cultivation of innovative thinking. Many of the educators there were some of the most forward thinking teachers I have met, focused on continuously improving their game and the value they were delivering to students. But classrooms that reflect that understanding remain a small minority. There are still far too many classes making these two big mistakes. Fortunately, I didn’t get booed off stage. Many of the educators in the room shared with me the awesome activities they do with their students to give them those butterflies. I’d love to hear more of your examples. How do you create uncertainty for your students or your employees to encourage entrepreneurial thinking? How do you give them butterflies in their stomachs and inspire them to solve problems?