For the past 60 years the financial services industry has been booming. Since 1980, year over year growth rates tripled and large entities flourished as smaller competitors crumbled. With low interest rates and new regulations, banks grew quickly as the rest of the marketplace piggybacked off their success. Savvy and risk-averse business men and women flocked to the industry; a career in financial services seemed like a safe bet.

Such a prolonged period of success is both a blessing and a curse. What made the financial services sector soar has also been responsible for nearly crippling it and taking our economy with it.

Here Comes Trouble

By succeeding for so many years – practically a lifetime in the business world – financial service firms remained virtually untouched while other industries were being disrupted. This made them dangerously comfortable. The strategies that once promoted growth stopped working in the same ways in today’s hyper competitive economy.

Ignoring changing tech and forgoing innovation for “guaranteed profits,” created the ideal environment for disruption.

In stark contrast to large, collateral backed loans, today’s financial innovators focus on small and specialized services like micro-loans, peer to peer lending, and small business lines of credit. The walls that once separated consumers from their finances gave way as startups like Square, Stripe, Coin, and Dwalla eliminate fees while providing new value.

Even business we typically wouldn’t consider part of financial services got into the game and expanded the field of competitors. Home Depot now offers loans for home remodeling projects. Starbucks and Chik-Fil-A created an app to speed-up payments and create financial loyalty. Walmart, Google, Apple Pay, Paypal are now viable forms of currency. The threats are ever increasing, and the financial services industry isn’t prepared to innovate and adapt.

The Expert Trap

While it's not fair to blame success for the impending doom, it is important to address the traps of reaching expert status. Yes, being really good at something, being an expert, can be what stifles innovation.

Enter: The Expert Traps. The silent and unseen killers of creativity and creation. When we feel like experts at what we do, the following traps await us:

Trap #1: We Stop Trying to Learn and Improve

Confidence in your skills is important. We’d all like our bankers, lawyers, and employers to be good at what they do. It’s a valid and honorable trait. However, when confidence grows without limits and exceeds its true skill, we lose the incentive to improve.

And this is what suffocated innovation in the financial industry. Big banks and firms were killing it (as in, they were raking in the profits). They were the best of the best; taking the time to reassess their offering would have only distracted them from continuing to win. Or so they thought.

This dangerous confidence convinces experts that it isn’t necessary to test and question the status quo. Hubris whispers “You know what you’re doing. You’ve got the track record to prove it.” While the beginner or the novice looks for opportunities to improve, the expert believes they will always be the best of the best, even as their stock prices plummet. This is one of the reasons they never saw the 2008 collapse coming.

Trap #2: We Don’t Examine Our Successes

Looking backward is a no-brainer when we lose. We can learn from our mistakes. Being a loser incentivizes us to examine what went wrong. But when we win, the opposite happens. Why look back when the future's so bright? By assuming the win was a result of perfect strategy and execution, you could ignore big opportunities to improve.

Winning is awesome. You will never find me happy with a loss. However, constant improvement and innovation demand we approach both losses and successes as a beginner. And a beginner never stops reflecting. If a beginner receives unexpected praise from a superior, the novice will ask, “What did I do and how can I do it again?”

So hold your head up high. Things are going well! Now turn around and take a hard look at how you got there. Can you repeat it? Was it luck? And most importantly, what could you have done better? Ask the hard questions and live to succeed another day.

Trap #3: We Play it Safe

Ah, to be the best of the best. It feels great to be at the top. So great, in fact, you’d rather not look down. Falling from such heights could be deadly.

For many in the financial industry, reaching the top triggered the instinct to survive. How can we avoid falling? What can we do to stay absolutely still and not compromise our position? Delaying or completely avoiding action feels like the safer option when you’ve fought so hard to get there.
But a beginner has nothing to lose. The beginner defaults to action, to responsible experimentation and seeing what happens. For the beginner, the opportunity to learn something new is more motivating than potentially making a mess. Failure is inevitable for a beginner, so why not try?

This doesn’t mean beginners experiment with reckless abandon, the goal is success after all. Beginners forgo guaranteed success in favor of growing their abilities. Experts hunker down and hold their position, even if it means ignoring an oncoming catastrophe.

 The Systematic Solution

Understanding the traps is a great first step. But it’s not enough. Research on cognitive biases shows that being aware of biases/expert traps doesn't do much to reduce their effects. All of the above traps happen subconsciously. Trying to “watch out” for the traps is just as difficult as “watching out” for how you’re breathing while you sleep.

The only way to control for the traps of success and expertise is to implement guardrails to keep your work on the right path. Here’s what I suggest:

1. Look for Negative Evidence

When you’re making a decision or choosing a strategic direction, assign someone to challenge your bias for a particular direction. The military calls them red teams, some in business call them a devil's advocate, and I call them provocateurs. Either way, find an individual or team of outsiders who have no vested interest in your project. They care enough to help, but not enough that they will be affected by the outcome. These individuals are in the best position to open your eyes to blind spots.

2. Retrospectives to Examine Success and Failure

Schedule a meeting at least once a year to look back at your project or company and examine the processes and results. Retrospectives create a structure for reflection, learning, and planning. No matter how well things are going, A Retrospective shows you how to correct glaring strategic mistakes that are invisible when you’re executing on day-to-day activities. I’ve used this process countless times and if you’re not sure where to start, you can use my 20+ page guide to running a successful Retrospective here.

3. Experiment

The most innovative companies are only so because of constant experiments. Experiments to see if they can improve things that are already working well. Experiment to find solutions to pervasive problems. Experimentation increases the velocity of decision making. It helps kill or change projects that aren’t working and lowers the cost of finding solutions. The best experiments use the scientific method, and if you’re not sure how to set up your next test, check out my template here.

Complacency is the enemy of innovation. The more comfortable you are, the less you work to improve and create new value. And complacency always takes its toll. As the pace picks up, the expert falls behind. Only those focussed on constant growth and improvement, even in good times, will be the ones to achieve sustained success.

Adopt the mindset of a beginner, build your retrospective strategy, and never stop fighting to improve. If you never settle, you will never peak! Let’s make the financial services industry boom for another 60, shall we?

Three years ago, I was coaching a successful company that was very confused about the failure of a recent product launch. When I was first hired, they told me they were convinced they had done everything by the book.

Before developing the product, the company held five focus groups with potential customers to make sure they were going in the right direction. After the product received positive feedback from each of the focus groups, the company spent hundreds of thousands of dollars in a little over a year to bring the product to market.

The product launch was disappointing to say the least. No one—not even the focus group participants who said they were interested—bought it. This experience is one of the most financially painful experiences in innovation, but it’s an experience that can be easily avoided.

In this example, the company was very excited about its new product idea. That excitement was obvious during the focus groups when moderators asked questions like, “Wouldn’t it be great if [blank]?” and, “Don’t you think [blank] is a big problem?”

Leading questions like these broadcast the answers you’re hoping to hear to your interview subjects. And as we’ve covered before, once your interview subjects understand what you want them to say, they are very likely to give you the answer you are hoping for, even if they don’t actually believe it.

Leading questions often result in false positives that waste money, time and opportunity.

After the company’s new product failed, they let me create an open-ended interview questionnaire to contact their target market. Not only did they find that the problem their product aimed to solve never existed, but they discovered that their product created new problems that they had not foreseen!

Use Open-Ended Questions to Identify Migraine Problems

Your customers’ natural inclination is to be nice, not honest.

Open-ended questions give your potential customers the option to respond very honestly and to fully explain the problems they’re facing (that your product may or may not solve). For example, you might ask someone, “Wouldn’t it be great if you never had to scoop your dog’s poop during walks?” If they’re a dog owner, chances are they will say yes. But if you ask them, “What have you done to solve that problem?” they will probably say they haven’t tried anything other than using bags they get from the grocery store.

When you use open-ended questions effectively, you’ll know you’ve hit a real migraine problem when the customer tells you it’s a problem without your prompting and then explains what specific actions they’ve taken to solve the problem.

Leading questions that broadcast the answers you want to hear:

“Are you…”
“Will you…”
“Won’t you…”
“Didn’t you…”
“Can’t you…”
“Would you…”
“Wouldn’t you…”
“If you could do [blank]…”

Open-ended questions start with less direct language:

“Tell me about a time when…”
“What do you think about [blank]?”

They also avoid biased language, like ”Do you agree that…” or “Don’t you hate it when…”

Comfort, Conversation and Active Listening

Not only do open-ended questions elicit more honest answers, they often make the customer more comfortable. No one wants to be interrogated, and few people want to be sold. Open-ended questions demonstrate to someone that you actually care what they think. It makes it obvious that your agenda is to learn, not to convince.

Unlike leading questions, open-ended questions turn the interview into a conversation. While your customer is talking—and if your questions are good, they may talk for a while—give them verbal and physical responses, like saying, “Uh-huh” or nodding your head. Talk as little as possible; you’re here to learn.

But in order to learn, you need to listen. Actively listen for any pain points, not just the one you expect to discover, and dig deeper when you’ve found one. You can do this by asking more direct questions:
“Tell me more about that.”
“What did you do to try to solve that problem?”
“What did you mean when you said [blank]?”
“Did you tell anyone about this problem?”
“Tell me about another time that you experienced something similar.”

Be Open, But Stay Focused

Even though open-ended questions can have a wide range of answers, the questions should still be focused. For example, if you have an idea for a client relationship management tool, you shouldn’t start by asking, “What are the three biggest problems in your life?” Instead ask the interviewee to tell you about how they keep in touch with potential and current clients. “Tell me about the tools that help you do that.” “Are there any that don’t work as well as you wanted?” After answering those few questions, you’ll have enough information to ask them to walk you through the process they followed for the last prospective client they targeted. “How did you hear about them?” “What did you do after the first contact?” “How did you stay in touch?”

By asking focused, unbiased, and open-ended questions during the initial interview—and digging deeper on pain points—you’ll have information you can immediately use to more effectively pursue the right ideas, and avoid ones that won’t work.

(Note: This is the second installment of a multipart series on conducting customer interviews and discovering true migraine problems.  And check out the first article in the series.)

Ok, you’re on the bandwagon.  You understand that you need to a get to a new level when it comes to understanding your customers and finding their migraine problems. You need to understand them better than they know themselves.

Here’s the thing: When you go out to interview customers, they are probably going to LIE to you.

I’ve lost count of how many companies I’ve seen fail while insisting that everyone they’ve talked to loved their idea. They think they’ve conducted the customer interviews they needed, but they didn’t understand one of the most important rules to customer interviews:

People’s natural inclination is to lie during interviews

Think about the last time a friend told you about their “brilliant” new idea that you thought was really dumb. Were you 100 percent honest with your friend? I bet you weren’t. You, and most people, would probably lie a little bit to get out of the awkward situation. It’s a lot like that for your customers. Their natural inclination is not to be completely honest with you during an interview. They want to figure out the answer you’re looking for and give you that answer so they can leave the conversation as soon as possible. Customers don’t lie to be malicious; it’s just the opposite.

Here are three main reasons customers will lie to you:

1.   They want to spare your feelings

They can clearly see how passionate you are about an idea, and they don’t want to be the one person to hurt your feelings.

When I was pregnant with my son and trying to decide on names, I conducted a fun experiment. I wanted to see what people really thought of the names my husband and I were considering, so when someone asked me what I was naming my son, I would give one of two responses. To one group, I said I wasn’t really sure, but I liked names like Branch, Major and True. To the other group I would say I was naming my son True, and that I was really excited about it. I quickly found that the two groups reacted very differently to my responses. When I told people I was still deciding, more than 90 percent would tell me they didn’t like any of the names I had picked. But when I acted excited about one name, over 90 percent of people told me they loved the name True and would go on and on about how unique it was. Obviously I wasn’t hearing the truth from everyone.

With my son’s name, I was just conducting an experiment. I didn’t really care what people thought. But, when you’re talking about starting a company, hearing the truth from customers is vital to your success. The problem is, if they can sense the answer you are hoping for, they will jump to give it to you.

2. They want to get out of the conversation

The key to understanding your customers real pain and current ways of solving existing problems is to get them talking.  You want to ask them open-ended questions that will elicit a lot more valuable details than the response to the specific question.  (More on open ended questions coming soon – make sure you are subscribed to get the article)

Now, think about the last time you got a call from a telemarketer trying to solicit a donation or get you to respond to a survey.  I bet with every question they asked, you went through a mental Rolodex of responses that might make the call end as soon as possible. You weren’t thinking of the real answers to their questions, let alone additional information they might find useful.  You were feeling uncomfortable and willing to say anything just to get it to end.

Most customer interviews are the same.  If you don’t do a good job of getting them comfortable and engaged before starting to ask them questions, they will just be looking for the fastest way to exit the conversation, and usually that means saying something like “That sounds awesome, I would probably buy that.”  They know that if they disagree with anything you said that you’ll want to argue with them, so they know that by agreeing with you and telling you how great your idea is, you’ll have no option but to end the conversation and go finish building your new product.

3. They want to seem smart

Many times when you present your idea, you’ll outline all the different ways it can benefit their lives, putting your potential customer in a situation where they look dumb if they disagree with you. For instance, you can tell them all of the money they are wasting each year from poor insulation in their home and all the benefits your product could provide to solve it for them.  What could they say?  “I don’t really care about all that money I’m wasting.” Or “I don’t really care about the environment.” Instead, they would likely try to save face by telling you how interesting it sounds and how they will think about buying your product.  They aren’t going to.  They just don’t want to look dumb, uncaring, or uninformed.

You have to trick people into telling the truth

Once you know that your customers are inclined to lie to you, your whole strategy should change. Your objective should be to trick them into telling you the truth, and to make it hard for them to lie by keeping them in the dark about what you’re really after. It’s the difference between asking someone’s opinion on a few ideas you’re considering and asking their opinion on one idea in which you obviously have a vested interest.

Interviewing customers is one of the most difficult steps in the entrepreneurial process. It’s not as exciting as launching a website or raising money. The truth can hurt too, especially if customers don’t end up liking your big idea. But, the only thing worse than not talking to customers is talking to customers who you don’t realize are lying to you.

What does it feel like to have a terrible business idea? The kind where people laugh behind your back as you walk away? Take a moment. Think about it.

It actually feels a lot like having the world's best business idea…right up until the moment when the market tells you that you don’t have anything anyone wants.

I meet with a lot of founders that don’t have any idea how to figure out whether they have the world’s best or dumbest idea.

After starting a number of companies of my own, and meeting with entrepreneurs at all stages of the startup journey, I’ve come to believe that the ability to differentiate good ideas from bad ones and massage bad ideas into profitable companies is like being physically fit. The vast majority of us aren’t born fit. We have to train, practice, and push ourselves to get good at it.

When I was in high school and college I completely ignored all forms of exercise. But I was lucky, I was a pretty skinny kid. People would always say to me, “are you a runner? You look like one!” and it happened so frequently that I came up with a funny quip back. I would say, “I run from danger!…and nothing else!”

So one day, I was walking my dogs, and I thought to myself, “Self, if there was danger on this street, could you in fact run from it?” I decided to test my assumption and I took off running. I sprinted as fast as I could. I sprinted for two whole blocks until I had to stop and throw up. And after I was done, I thought, "Man, danger would have kicked my butt."

That day I learned that it’s pretty easy to make bad assumptions. Just because a lot of people tell you something is true -- that you look like a runner -- doesn’t necessarily make it so.

Your startup idea is no different than my perceived ability to run from danger. The only way to prove or disprove your idea’s likelihood for success is to test your assumptions. And whether or not this current idea makes it, you’ll be developing the muscle memory to validate future ideas.

No innovator fails because they couldn’t build their product.  They fail because no one found value in what they built.

Here’s what usually happens when someone gets a new idea:

  1. Their mind starts racing with all the possibilities of what it could turn into, the impact it could have on the world, and all the money it could generate.
  2. Next, they begin executing on their idea.  All of their focus is internal, working hard to come in on time and on budget.  They need to make this product amazing. First impressions are everything!
  3. Then, they work on marketing.  The product needs a catchy name and logo.  It needs beautiful collateral both online and offline. This has to look innovative!
  4. Finally, they take the product to market and, more often than not, get an extremely lukewarm reception.  So they blame marketing, and they blame the product for not having enough features.  Sometimes they bring new members onto the team and start the cycle from scratch.  Sometimes they just run out of resources and the project gets mothballed.

Sadly, this is the typical cycle of innovation.  Most new initiatives fail.

But successful innovators know that they have a very powerful tool at their disposal to significantly decrease the risk of innovation: experimentation.

Experiments are small bets that you make to see if what you believe to be true is actually true - to see if your predictions about the customer and the market are right.   It’s something small that you do today to prove that you are spending time and money on the right things . . . on building something that people will buy.

In All in Startup, the reader meets Owen after he has followed the very process outlined above and wasted hundreds of thousands of dollars and a year of his life building something that wasn’t meeting any of his projections.  What could he have done before he committed his available resources to executing his plan, to make sure customers would be waiting for him when the product was ready?  He should have run some experiments!

But the hardest thing about experiments is running them correctly.

Here are some guidelines about what a good experiment looks like.

To run a good experiment, you need to determine and document these five elements before you begin:

1. The Goal

What is it that you’re trying to learn? Or what are you trying to prove? What are the riskiest assumptions you’ve made about your idea? For Owen, his riskiest assumption was that people would buy half-priced, used bicycles online.

2. The Hypothesis

Your high school science classes taught you what you need to know for this part. This is a statement that you’re trying to prove true or false through the experiment.  The result will be a “yes” or a “no” so you need to phrase your hypothesis appropriately.

For example, a good hypothesis would be based on this setup: “If I do this action, then this outcome will happen.”

The key is to make sure that your Hypothesis helps you get closer to the Goal you outlined above and reduce the risk of your riskiest assumption. The hypothesis can help you test whether you have identified the right customer segment, whether your target customers actually have the pain point you think they do, whether they perceive enough value in your solution to buy it, whether your solution actually solves their pain point, whether you’ve identified the right channels to target your customers, whether your supplier cost estimates are accurate, whether you’ve chosen the right price point for your product, etc.

The biggest mistakes in putting together a hypothesis include 1) Creating a hypothesis that isn’t measurable and lacks a clear beginning and end (ex. “People want to eat healthier”); 2) Creating a hypothesis that isn’t about a specific group of customers (ex. Everyone wants a car that gets at least 30mpg); 3) Creating a hypothesis that doesn’t help you reach your goal (ex. “If I send this survey to 100 people, 10% will fill it out”); or 4) Creating a hypothesis that isn’t refutable, meaning that it’s difficult or impossible to prove it false (ex. “Restaurants want more customers”).

Some example hypotheses Owen could have created:

“If I put up a landing page to sell half-priced, used bicycles, then 5% or more of the website visitors will preorder the bikes.” – Testing value

“If I set up a booth at a bike race for one day, I will sell at least 10 bicycles” – Testing customer segment

“If I spend $500 on facebook ads targeting people with cycling listed as an interest, at least 5 people will click on the buy button and enter their payment information” – Testing customer segment and marketing channel

“If I call 20 bike shop owners, at least 3 will come to my shop for a one-hour meeting to find out more about my bikes and whether they want to sell them in their store” – Testing distribution channel.

3. The Subject

Who are you targeting with the experiment?  How are you filtering who will participate and who won't? For instance, if Owen puts up a landing page to see if people interested in road bikes will want to buy his bikes, what kind of information is he gathering on the landing page to make sure that the right people are seeing his messaging before he decides whether it's working or not?

Tip: If you are having trouble limiting your target subject for the experiment, try first listing out people who wouldn't fit into your target subject. I.e., for Owen it's people who want to buy a $100 bicycle at Walmart or Target or perhaps people who are interested in roadbikes but have never actually purchased one because they think they are too expensive.

4. The Logistics

How are you going to conduct your experiment? What’s the time period? How do we know when the experiment has started and has finished? How many people will you target? Who will carry out the experiment?

A key question to ask yourself here is: is this the least amount of time and effort I can spend to test this hypothesis? Remember, this is supposed to be a small bet you can afford to lose.  Too many people think their experiment is building a lighter version of the final product – taking 6 months to put together.

You should be able to run your experiment in under 2 weeks.  I will frequently push my innovators to come up with a hypothesis and figure out a way to start the experiment within 24 hours.

5. The Currency

Something of value that the subjects of your experiment have to give you in order to prove whether your hypothesis is true.

The key is that it be something painful for them to give up in order to demonstrate their sincerity.  This can include anything from money to time to a commitment of certain resources.  Basically anything that demonstrates customers will be willing to part with something they value in order to obtain the product or service you’re offering. You want to make sure that they aren’t just trying to be nice to you or lying about their intent for some other reason.

When you are trying to de-risk an idea, the worst kind of evidence you could gather is a false positive (i.e., perceived interest from people who don’t actually see value in your product) because it gets you excited about moving forward and going All In on the idea. Simulate a world where your product already exists, and see if your customers will give you the currency you think you deserve.

In the Owen hypothesis examples above, Owen is asking for specific actions, a commitment of money or commitment of time to demonstrate whether he’s identified the right customer segments, marketing channels and distribution channels.

6. The Success and Failure Criteria

Before you begin your experiment, it’s important to define what success and failure will look like.  If success is having 25 percent of customers give you currency, what does it mean when only 15 percent provide it? Has your experiment failed?

You need to set up these parameters before you begin the experiment, so that you’ll objectively understand the outcome and not be forced to debate what the results mean with your team.

Additionally, a friend of mine, Justin Wilcox, suggests that innovators write out two separate plans of action to pursue depending on whether the experiment reveals the hypothesis to be correct or incorrect. He emphasizes that this should be done before conducting the experiment.  Some people have such a hard time deciding what to do if an experiment doesn’t go as they had planned that they end up making up a justification of why it was a success and allowing themselves to keep moving forward on their idea.
OK.  Those are the elements you need to determine and document before you begin your experiment.  And while this experiment framework can seem cumbersome, remember, it is your single greatest asset in reducing the risk that goes with creating something new.

I’ve been thinking about my first motorcycle accident. I’m not counting the time when I drove my Kawasaki Ninja to my friend Eric’s house and, while showing it off, fell over and needed his help to pick it back up. No, this was a lot more serious

I was attempting to make a left turn at a light and I think I must have slipped in something greasy on the ground, lost control and ended up running the bike into the curb. As I hit the curb, I flew off the bike and landed on my head and shoulder. My head was ok, but I really hurt my rotator cuff in my right shoulder. It’s been acting up a lot lately, so I’ve been reminiscing over the accident.

Riding a motorcycle is really dangerous. And some of the navigation is extremely counter intuitive. For instance, if you want to make a left turn while traveling above 5mph, you need to push the handlebars in the opposite direction (to the right) in order to turn left. It feels really weird when you first do it. And it takes a lot of practice to get used to it. I can’t really explain to you why it works, but I know that it’s counter to everything your gut instincts tell you.

We’ve all heard the story about the boiling frog. That if you put a frog into a pot of boiling water, it will instantly jump out, but if you set the frog in a pot of cold water and slowly turn up the heat, it will gradually cook to death. (For the record, this is one experiment I’ve never conducted. No frogs were harmed in the writing of this post.) The frog story is how I think about motorcycle riding and entrepreneurship.

Motorcycles are obviously dangerous. They're like the pot of boiling water. But entrepreneurship can be just as detrimental to your health as riding motorcycles; it’s just a lot more subtle. I’ve seen people lose their life savings, their relationships, and their health struggling to “make it” as an entrepreneur. To some extent, I’ve experienced each of these stresses myself through my entrepreneurial journey. It’s just hard to see when you are in a pot where the water is getting hotter and hotter, where the amount of risk you are getting yourself into is growing and growing.

And as is the case with riding a motorcycle, there are several counterintuitive principles you have to master to succeed as an entrepreneur. Just like pushing the handlebars in the opposite direction of the way you want to turn, there are things you need to do to be successful as an entrepreneur that feel totally unnatural, counter to your instincts, and downright hard to explain.

The most important of these counterintuitive principles is that people buy solutions to their problems, not products and services.

The more serious the pain you can solve, the more it means you can charge for your product. You can bank better margins, enjoy much shorter sale cycles, and spend a lot less on sales and marketing because your customers will be evangelizing the product on your behalf through powerful word of mouth testimonials.

Sounds pretty good, right? But unfortunately, 95% of entrepreneurs don’t follow this simple rule. That number is probably even higher than 95%, but I want you to keep reading and not get hung up on it.

It’s not that most entrepreneurs deny that solving problems is important. It’s that they go out of business because they build solutions for problems that don’t actually exist.

Here’s what usually happens:

An entrepreneur will get an idea and start thinking of all the possibilities of what it could turn into, the impact it could have on the world, and all the money it could generate.
Next, they will build the product. They spends a lot of time and money trying to build the most comprehensive version of it, rarely showing it to anyone because they want it to be perfect before potential customers see it. First impressions are everything!
Then, they brand the idea. They spend time and money developing a catchy name and a logo, purchasing a domain, building a website, creating marketing materials, etc. This has to look professional!
Finally, they go out looking for customers and, more often than not, strike out big time, causing them to realize that something is wrong with the initial idea. They revisit the idea and start brainstorming how to make it better. And then they repeat step one through four all over again, spending a lot of time and money, without making any forward progress.

This is the startup loop of despair. It can last anywhere from a few months to a few years, and it usually ends when the entrepreneur finally runs out of money. When they are fully cooked!

But successful entrepreneurs know that the startup loop of despair is completely avoidable. They know that once you come up with a great idea, the very next step should be to find potential customers and determine if your product is even worth building.

There is only one way to diagnose whether your idea solves a real problem. You have to conduct a Problem/Solution Fit Test.

A lot has been written about this process, but many still have trouble applying it because it’s so counterintuitive. It feels weird and awkward and it’s hard to know when you are doing it right. So I wanted to break it down and try to do my best to explain it.

What exactly are we testing?

Two things.

1. The Problem – Does a specific group of customers have a migraine problem?

2. The Solution – Does my solution solve the problem?

How are we testing these two things?

We actually use different tools for each part of the test.

1. Diagnosing migraine problems is most effectively done through customer interviews. Believe it or not, customer interviews are extremely tricky because both you and the customers are coming into the conversation with a number of biases, so we have to conduct them in a very specific way to make sure we are getting honest feedback that we can use.

2. Solution testing is best done through objective experimentation. Humans are terrible at predicting the future. So we can’t really interview our customers to understand if the solution will work. We have to find a way to simulate the future and objectively test how customers will actually behave.

Conducting the Problem/Solution Fit Test before building your product will guarantee that you will build a product people actually want by figuring out which features and benefits are the most valuable. Above all, this means your startup will actually generate revenue.

But what about Henry Ford and Steve Jobs?

I get asked this question a lot. You’ve heard the quote, Henry Ford once said that if he were to ask people what they wanted, they would have said “faster horses.”

He’s totally right. You can’t ask people what they want. They are terrible at predicting the future and sometimes they plain out lie. But you can ask them about the past or the present. About current problems they are facing. And if Henry asked potential customers if they had pain about trying to find transportation to their jobs or to visit friends or family, they would have talked his ear off about all the different things they’ve tried to solve the problem.

Ok, and what about Steve Jobs? Most people think he never tested any of his products. He must have predicted what we would want!

Yes, Steve Jobs is a genius. There is no doubt about that. He beautifully combined art and technology, but not all his guesses about what the people wanted were right. The original Macintosh that Jobs created was a failure. It didn’t have a fan because Jobs thought it distracted from the calm of the computer and ended up frying all of the internal parts. The Lisa computer he created was also a failure. His NEXT company, created another personal computer that was a huge flop. But he had something like a hundred million dollars from his initial success and the ability to make lots of big bets without going broke.

If you don’t have tens of millions of dollars you can afford to lose just trying out different startup ideas, then you’re not Steve Jobs.

But here’s some good news. You can use the Product/Market Fit Test to significantly de-risk your bets before you go all in. The best way to de-risk an idea is to make sure people want it before you spend too much time and money creating it. If you aren’t solving a migraine problem, you’ll need to spend a lot of time and money educating people about your existence and and convincing them that they can benefit from your product. It’s very expensive to convince people that they need or want something that they really don’t.

When All In Startup first came out, I put together a list of customer interview rules to help entrepreneurs conduct the Problem/Solution Fit Test. You can find the original list here. Since then, I’ve received hundreds of follow up questions. So I’ve decided to spend the next few posts diving deep into the process of interacting with your customers.

Whether you are an entrepreneur with a crazy idea or a large company with thousands of existing clients, the upcoming articles on interacting with your customers will help you significantly de-risk your business.

You will learn:

How to get honest feedback from your customers
Which questions are the most valuable to ask
How to conduct small experiments to make sure customers are interested in your product before you invest too much time and money creating it

To make sure you receive these posts when they become available, subscribe here. And please either send me an email or add a comment to this post to let me know specific questions you have about the process to make sure I answer them.

I have a small confession to make. As I thought about the bike accident over the last 15 years, I’ve always said there must have been some grease in the road to slip me up. Some kind of discharge from a car that made the road hazardous. But as I was writing this post and thinking about my experience level at the time, I think that it’s much more likely that I hesitated. That I went with my gut and what my instincts were telling me to do instead of following the framework that I knew would be effective.

I know how scary it is to start something new. I know what it feels like to put everything on the line. I’ve felt the pain of what happens when your instincts take over. My objective is to help both you and me find a way to follow the counterintuitive rules of the road to get us all home safe.

Thanks for reading.

Corporate innovation is all the rage these days. With plenty of money and time at their disposal, employees have ample resources to create the next big thing. But before they commit too many resources to an idea, these intrapreneurs can learn a lot from successful entrepreneurs who operate with fewer resources.

Innovation within a corporation comes down to two factors:
1.     How employees generate ideas
2.     How employees determine which ideas to pursue

1. How employees generate ideas (problem-oriented vs. solution-oriented ideas)
Do you remember the Segway? The Segway is a two-wheeled, battery-powered machine that makes even the most popular kid in school look like he sits alone at lunch.  What I’m saying is you look dorky riding the thing.  There’s something about you not putting in much effort while standing and gliding down the street that just makes people uncomfortable.

When the Segway first came out, it garnered lots of attention. Steve Jobs predicted the Segway would be bigger than the PC. John Doerr, a prominent venture capitalist who backed Netscape and Amazon, said it would be bigger than the Internet. With this level of hype, the company raised more than $90 million. But it’s grand unveiling was a huge disappointment. It took the company its entire first year in business to sell the number of Segways it predicted it would sell every two weeks.

The company had generated a solution-based idea. The Segway was a shiny, new technological advancement…that nobody wanted. The product didn’t solve a problem for the customer. Most companies that are unhappy with the results of their innovation programs are generating these same kinds of solution-based ideas. They start with a technological advancement or a brain storming session that produce a product, and then they try to figure out who might want it.

They’re so excited to have created something new that they forget that no one necessarily asked them to create it in the first place. They just assume that if they build it, the customers will come. Unfortunately, that seldom happens.  What innovative employees – or intrapreneurs – do differently is come up with solutions inspired by their customers’ problems.

Instead of taking stabs in the dark and guessing what people might like, why not go directly to your customer and figure out for sure? When Paul Buchheit at Google created Gmail, he did just that.  He came up with a problem-based idea. Instead of modeling his email platform after others on the market (looking at competitors to see what features and benefits to include), Paul listed out all the problems he thought the existing solutions created for users.  The existing email platforms had limited storage space, were hard to search through and were slow to load data. By designing Gmail around these problems, Paul created something that generated a lot of value to email users.  He initially designed the gmail platform for his own use, but when others in the company saw it, they begged him to let them have access to the functionality he created.  And when Gmail became public, users flocked to the service because word of mouth marketing was so powerful at explaining its value.

Word of mouth marketing only works when people feel compelled to spread the word – and that compulsion is borne out of feeling that a product has solved a tangible problem. Gmail wasn’t just different to be different.  It was different in a way that generated a lot of value for its customers. So much value that they wanted to tell people about it at school or work.

Like Gmail, most successful products are built to solve problems. These problem-based ideas can only be found through customer interaction. You know you have a potentially great idea when you identify a group of customers who are ready, willing and accessible. Ready customers have identified a problem and are interested in fixing it, willing customers have taken steps to fix the problem in the past and have a budget to address the problem, and accessible customers are people you can easily reach. When you seek out this trifecta of customer interest for a problem-based idea, your product will practically sell itself.

2. How employees determine which ideas to pursue 
Webvan is one of the most famous examples of a seemingly good idea that flopped. In 1996, Webvan launched an online grocery delivery business on the theory that people would pay money to avoid the hassle of grocery stores. Webvan spent hundreds of millions of dollarsbuilding an infrastructure (warehouses, technology, sales/marketing) based on a business plan that should have worked. But it never took off, and eventually the company went bankrupt in 2001. The online news site CNET went on to name Webvan the worst dot-com failure in history.

The Webvan story follows a plan-based approach to business that is commonly taught in business school. You write a business plan, see whether the numbers add up, and predict the success of a business based on this creative writing exercise. The plan-based approach seldom works because innovation is iterative. Your first business plan won’t be perfect, but most people don’t give themselves enough resources to make necessary pivots. Instead, many companies spend all their money to build a perfect product that is often a failure.

Entrepreneurs with the most positive results, however, know their company will be a success before their product ever hits the shelves. Instead of trying to predict the future, these entrepreneurs try to play detective by using an evidence-based approach to innovation. Theevidence-based approach determines customer demand and value of a product through experiments with actual customers. Simply by using this approach, the creators of Webvan could have launched a successful company or, at worst, abandoned the idea and saved hundreds of millions of dollars.

Zappos, an online shoe retailer founded in 1999, is the opposite of Webvan. Before the company spent a dime on building its infrastructure, it made sure it had customers by verifying a willingness among customers to buy shoes online. In the beginning, founder Nick Swinmurn would take pictures of the shoes available at local stores and post them on Zappos’s website. After a customer made a purchase online, Nick would return to the local store, buy the pair of shoes at full price and ship the shoes out manually. Before Zappos had built a single shipping warehouse, it was able to prove that its business model would work. This evidence-based approach is what prevented Zappos from becoming Webvan.

A few years after Webvan closed its doors, Zappos was acquired by Amazon for $1.2Billion. Swinmurn’s early experiments, while time consuming, obviously paid off. It’s important to verify that customers will actually pay for your product. A customer can easily give you an email address or a verbal agreement to buy your product, but real “currency” is different. When a customer gives you something that pains them, like money, time or an endorsement, you have strong evidence that your product is highly valued.

With everything on the line, entrepreneurs with access to little capital are under heightened pressure to build a moneymaking product on the first try. The best entrepreneurs reduce risk and expense by generating ideas that solve a problem and verifying those ideas with customer-backed evidence. This approach is often used out of necessity by entrepreneurs, but successful employee-driven innovations result from mimicking this method within the corporate environment. A problem-based idea and evidence-based approach is essential for innovation anywhere. Whether you have one employee or 5,000, the pain of innovation can be both eased and energized by following the example of successful entrepreneurs.

Launching new products and services is the only way to stay competitive in the new economy. Unfortunately, most companies have a process in place that stifles innovation.

I recently gave a TEDx talk titled "Our Approach to Innovation is Dead Wrong" about why our traditional thinking with regard to launching new products or initiatives is actually killing those new ideas before they ever have a chance to succeed.

So how does your organization stack up?  

What's working well and what do you think isn't working?

Looking forward to your thoughts.

A few weeks ago, at the National Association of Community College Entrepreneurship (NACCE) Conference, I took part in a panel discussion on the subject of preparing our future workforce for entrepreneurial thinking.

As companies increasingly express an interest in creative, innovative employees, one of the hottest questions in entrepreneurship education has become, “How do we prepare graduates to solve real problems and find opportunities for their employer rather than just show up to a job?”

The first question I received on the panel was “how are we doing as a community of entrepreneurship educators in preparing this future workforce?”

I was quiet for a few seconds. Frankly, I was nervous about what would happen if I said the first thing that came to my mind. I prepared myself to get booed off the stage and said, “I think we’re doing a terrible job. In fact, I think that we are actually killing innovative and entrepreneurial thinking through the classes that we teach.”

They didn’t erupt in boos. They let me continue.

1. Students are pitching to professors, not customers.

In many entrepreneurship classrooms, professors make judgment calls about good ideas and bad ideas. Sometimes students pitch their ideas in business plan competitions to panels of experts that decide the value of a potential company. In the real world, logic and expertise can’t predict customer behavior. Even the top professional startup investors are wrong about predicting which startups will be successful 90% of the time. The only way to create a successful company is to discover what customers actually want through direct interactions, not assumptions. Instead of making judgments, we should push students to interact with potential customers and conduct experiments to see if they can simulate sales. If a student is required to appease an internal source of authority, like a professor, for the sake of their grade, then they won’t learn to respect the true source of authority, the customer.

2. Classroom work isn’t giving students butterflies in their stomachs.

Creating a successful company is not as simple as checking things off a to-do list. Yet many professors still give students a list of tasks—create a business plan, interview an entrepreneur, read a book—as if there is an easy roadmap to building a million dollar company. Entrepreneurship is an emotional roller coaster. It’s scary and exciting all at the same time, and, above all else, it’s fraught with uncertainty. If teachers aren’t giving students butterflies in their stomachs – making them feel overwhelmed or uncomfortable – then they aren’t preparing students for the challenges of entrepreneurship. And they certainly aren’t teaching them to develop entrepreneurial thinking. Instead, teachers who don’t push students out of their comfort zone are simply reinforcing traditional 9 to 5 employee thinking. There’s nothing wrong with a 9 to 5 employee, but it’s not the mindset companies are currently looking for as they interview business school graduates.

If you want to give your students real butterflies, then give them an objective and ask them to figure out how to achieve it. Better yet, send them out on some real experiential activities, like the activities suggested in this curriculum. There is no doubt that uncertainty in a classroom makes students uncomfortable. They like having tasks that make it easy for them to walk the road to success. But, if we keep teaching students entrepreneurship that only works in a classroom, and not in the real world, we aren’t doing them any favors. In fact, we’re setting them up to fail themselves and their employer.

It is our responsibility to push students to fight through difficulty and uncertainty. If your students are uncomfortable, that’s a good thing. When they enter into the workforce, they will be armed with an entrepreneurial mindset, arguably the most important key to success in business over the next 20 years. The great thing about the attendees of the NACCE conference is that most of them already know these two potential barriers to the cultivation of innovative thinking. Many of the educators there were some of the most forward thinking teachers I have met, focused on continuously improving their game and the value they were delivering to students. But classrooms that reflect that understanding remain a small minority. There are still far too many classes making these two big mistakes. Fortunately, I didn’t get booed off stage. Many of the educators in the room shared with me the awesome activities they do with their students to give them those butterflies. I’d love to hear more of your examples. How do you create uncertainty for your students or your employees to encourage entrepreneurial thinking? How do you give them butterflies in their stomachs and inspire them to solve problems?

The data you choose to measure to test the success of a project is very important. If you measure vanity metrics instead of measuring real value based on the strategic goals of the organization, you could end up with a warehouse full of cupcake supplies.

The metrics you choose to measure the success of any project are very important. If you measure the wrong information, you might end up with a false positive. You might think a project is going well only to figure out later that it was a waste of time, money and effort.

But, if you measure the right metrics and approach Value Metrics like a scientist, you'll be able to weed out wasted effort much faster and more efficiently.

The Cupcake KPI

When Quantitative Strategist Brandon Ritzo began working for a home loan company in 2012, each customer was receiving a custom cupcake to remind them to fill out their loan paperwork. For $25 a pop, the cupcakes were made and shipped overnight to their destinations. Brandon was tasked with figuring out if the cupcakes scheme benefitted the company's bottom line.

Choose the right Value Metrics

You have to choose Value Metrics very carefully because if you end up using the wrong ones, you're going to see value where there isn't any. For example, in the cupcake experiment, people responded very positively to the cupcakes on social media by posting pictures and sending virtual thank you notes. Although interacting with customers on social media is a form of engagement, it's also a vanity metric. “Likes” and sales are two very different things.

The real Value Metrics of the cupcake scheme, Brandon thought, should be whether or not the cupcakes resulted in more customers closing loans. After all, closed loans directly added value to his employer's bottom line.

Set a control group.

To know if an initiative is working, you have to have a control group. When the cupcake creators thought up their idea, they decided to buy the supplies in bulk and rent a warehouse to store the excess. Brandon put some of the extra cupcake supplies to good use by differentiating the cupcake group from the non-cupcake group. Without a control group, you can't test the results of your efforts.

Run your experiment and interpret the results.

When Brandon's team started to compare the control group to the cupcake group, they discovered that the cupcakes were resulting in more customers turning in their paperwork! To a less scientific person, this might indicate it's time to throw an office party and rent another cupcake warehouse.

But even though signed paperwork was the Value Metric the group originally set—the cupcakes came with a note reminding people to send in their paperwork—it turned out to be a Vanity Metric, a piece of data that makes you feel really good, but is actually concealing bad results.

Brandon's team found that the cupcakes had no effect on the real Value Metric: closing loans.

Not only were the cupcakes, shipping and warehousing costing the company money, but processing the increased amount of paperwork resulting from the cupcakes was costing the company money without any benefit to the bottom line.

“If we get extra paperwork, but we aren't getting to the final stage, everyone is worse off. We've wasted everyone's time and created more work,” Brandon said. “When you apply that logic, you're not only spending money on cupcakes but you're also costing the company money in overhead.”

Know when to stop.

After discovering the real cost of the cupcake experiment, Brandon's employer stopped shipping cupcakes. Even if it's hard to admit that an idea may be a bad one, the only other option is to trudge through bad ideas, accept sub-par results, and ultimately have to admit an even bigger failure down the road.

Apply this scrutiny to everything.

After shutting down the cupcake scheme, Brandon began looking for other projects that weren't going well. The project managers knew something was broken, so Brandon's team would come in, break the process apart entirely and see where they could deliver more value.

For example, the company also provides free credit counseling for people who don't qualify for a home loan. “We were drowning in clients for this service,” Brandon said. People would sign up, but then many would almost immediately stop participating. For this example, the Value Metric was whether or not the customer was able to improve their credit score to the level required by most banks to get a mortgage.

The conventional wisdom was to work with wealthier customers who would have the resources to dig themselves out of the hole, or to work with customers closest to achieving their goals because there was less distance to cover. But Brandon's team was able to isolate a much more influential factor: how well these customers lived below their means. So, they developed some methods to identify these customers and encourage counselors to focus on these customers first.

From there, Brandon and his team streamlined ways to test ideas as quickly as possible. From the initial results, they could slowly scale up successful projects before putting the full weight of the company behind something that might not add value to the organization.

Continue to measure Value Metrics.

Even if Brandon's team has improved an initiative, they never stop measuring Value Metrics. For example, after the credit counseling experiment, 2 percent of all call volume continued to follow the previous standards. Even improved initiatives aren't perfect. You should still measure the right Value Metrics, have a control group, and keep experimenting and refining to avoid perpetuating errors from the past.

Ready to join the curious community?
linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram