What does it feel like to have a terrible business idea? The kind where people laugh behind your back as you walk away? Take a moment. Think about it.
It actually feels a lot like having the world's best business idea…right up until the moment when the market tells you that you don’t have anything anyone wants.
I meet with a lot of founders that don’t have any idea how to figure out whether they have the world’s best or dumbest idea.
After starting a number of companies of my own, and meeting with entrepreneurs at all stages of the startup journey, I’ve come to believe that the ability to differentiate good ideas from bad ones and massage bad ideas into profitable companies is like being physically fit. The vast majority of us aren’t born fit. We have to train, practice, and push ourselves to get good at it.
When I was in high school and college I completely ignored all forms of exercise. But I was lucky, I was a pretty skinny kid. People would always say to me, “are you a runner? You look like one!” and it happened so frequently that I came up with a funny quip back. I would say, “I run from danger!…and nothing else!”
So one day, I was walking my dogs, and I thought to myself, “Self, if there was danger on this street, could you in fact run from it?” I decided to test my assumption and I took off running. I sprinted as fast as I could. I sprinted for two whole blocks until I had to stop and throw up. And after I was done, I thought, "Man, danger would have kicked my butt."
That day I learned that it’s pretty easy to make bad assumptions. Just because a lot of people tell you something is true -- that you look like a runner -- doesn’t necessarily make it so.
Your startup idea is no different than my perceived ability to run from danger. The only way to prove or disprove your idea’s likelihood for success is to test your assumptions. And whether or not this current idea makes it, you’ll be developing the muscle memory to validate future ideas.
No innovator fails because they couldn’t build their product. They fail because no one found value in what they built.
Here’s what usually happens when someone gets a new idea:
Sadly, this is the typical cycle of innovation. Most new initiatives fail.
But successful innovators know that they have a very powerful tool at their disposal to significantly decrease the risk of innovation: experimentation.
Experiments are small bets that you make to see if what you believe to be true is actually true - to see if your predictions about the customer and the market are right. It’s something small that you do today to prove that you are spending time and money on the right things . . . on building something that people will buy.
In All in Startup, the reader meets Owen after he has followed the very process outlined above and wasted hundreds of thousands of dollars and a year of his life building something that wasn’t meeting any of his projections. What could he have done before he committed his available resources to executing his plan, to make sure customers would be waiting for him when the product was ready? He should have run some experiments!
But the hardest thing about experiments is running them correctly.
Here are some guidelines about what a good experiment looks like.
To run a good experiment, you need to determine and document these five elements before you begin:
What is it that you’re trying to learn? Or what are you trying to prove? What are the riskiest assumptions you’ve made about your idea? For Owen, his riskiest assumption was that people would buy half-priced, used bicycles online.
Your high school science classes taught you what you need to know for this part. This is a statement that you’re trying to prove true or false through the experiment. The result will be a “yes” or a “no” so you need to phrase your hypothesis appropriately.
For example, a good hypothesis would be based on this setup: “If I do this action, then this outcome will happen.”
The key is to make sure that your Hypothesis helps you get closer to the Goal you outlined above and reduce the risk of your riskiest assumption. The hypothesis can help you test whether you have identified the right customer segment, whether your target customers actually have the pain point you think they do, whether they perceive enough value in your solution to buy it, whether your solution actually solves their pain point, whether you’ve identified the right channels to target your customers, whether your supplier cost estimates are accurate, whether you’ve chosen the right price point for your product, etc.
The biggest mistakes in putting together a hypothesis include 1) Creating a hypothesis that isn’t measurable and lacks a clear beginning and end (ex. “People want to eat healthier”); 2) Creating a hypothesis that isn’t about a specific group of customers (ex. Everyone wants a car that gets at least 30mpg); 3) Creating a hypothesis that doesn’t help you reach your goal (ex. “If I send this survey to 100 people, 10% will fill it out”); or 4) Creating a hypothesis that isn’t refutable, meaning that it’s difficult or impossible to prove it false (ex. “Restaurants want more customers”).
Some example hypotheses Owen could have created:
“If I put up a landing page to sell half-priced, used bicycles, then 5% or more of the website visitors will preorder the bikes.” – Testing value
“If I set up a booth at a bike race for one day, I will sell at least 10 bicycles” – Testing customer segment
“If I spend $500 on facebook ads targeting people with cycling listed as an interest, at least 5 people will click on the buy button and enter their payment information” – Testing customer segment and marketing channel
“If I call 20 bike shop owners, at least 3 will come to my shop for a one-hour meeting to find out more about my bikes and whether they want to sell them in their store” – Testing distribution channel.
Who are you targeting with the experiment? How are you filtering who will participate and who won't? For instance, if Owen puts up a landing page to see if people interested in road bikes will want to buy his bikes, what kind of information is he gathering on the landing page to make sure that the right people are seeing his messaging before he decides whether it's working or not?
Tip: If you are having trouble limiting your target subject for the experiment, try first listing out people who wouldn't fit into your target subject. I.e., for Owen it's people who want to buy a $100 bicycle at Walmart or Target or perhaps people who are interested in roadbikes but have never actually purchased one because they think they are too expensive.
How are you going to conduct your experiment? What’s the time period? How do we know when the experiment has started and has finished? How many people will you target? Who will carry out the experiment?
A key question to ask yourself here is: is this the least amount of time and effort I can spend to test this hypothesis? Remember, this is supposed to be a small bet you can afford to lose. Too many people think their experiment is building a lighter version of the final product – taking 6 months to put together.
You should be able to run your experiment in under 2 weeks. I will frequently push my innovators to come up with a hypothesis and figure out a way to start the experiment within 24 hours.
Something of value that the subjects of your experiment have to give you in order to prove whether your hypothesis is true.
The key is that it be something painful for them to give up in order to demonstrate their sincerity. This can include anything from money to time to a commitment of certain resources. Basically anything that demonstrates customers will be willing to part with something they value in order to obtain the product or service you’re offering. You want to make sure that they aren’t just trying to be nice to you or lying about their intent for some other reason.
When you are trying to de-risk an idea, the worst kind of evidence you could gather is a false positive (i.e., perceived interest from people who don’t actually see value in your product) because it gets you excited about moving forward and going All In on the idea. Simulate a world where your product already exists, and see if your customers will give you the currency you think you deserve.
In the Owen hypothesis examples above, Owen is asking for specific actions, a commitment of money or commitment of time to demonstrate whether he’s identified the right customer segments, marketing channels and distribution channels.
Before you begin your experiment, it’s important to define what success and failure will look like. If success is having 25 percent of customers give you currency, what does it mean when only 15 percent provide it? Has your experiment failed?
You need to set up these parameters before you begin the experiment, so that you’ll objectively understand the outcome and not be forced to debate what the results mean with your team.
Additionally, a friend of mine, Justin Wilcox, suggests that innovators write out two separate plans of action to pursue depending on whether the experiment reveals the hypothesis to be correct or incorrect. He emphasizes that this should be done before conducting the experiment. Some people have such a hard time deciding what to do if an experiment doesn’t go as they had planned that they end up making up a justification of why it was a success and allowing themselves to keep moving forward on their idea.
OK. Those are the elements you need to determine and document before you begin your experiment. And while this experiment framework can seem cumbersome, remember, it is your single greatest asset in reducing the risk that goes with creating something new.