Lean Startup Build-Measure-Learn Loop Planning

The Build-Measure-Learn loop is the heart of Lean Startup and every iteration through the loop requires some degree of planning. Build-Measure-Learn iteration planning is similar to Agile iteration planning where the startup team plans the activities and identifies the expected results for the next iteration of the Build-Measure-Learn loop.

 

Build-Measure-Learn Iteration Planning Session

 

The purpose of the Build-Measure-Learn iteration is to test some unknown aspect of the business. This might be an experiment to learn about customers, the product, sales, marketing, support, packaging, or something else that the business feels will increase the value of the product or increase the growth of the business. Whether the business is in pre- or post-Product/Market Fit, the Build-Measure-Learn iteration serves the same purpose: learning to improve and grow the business using scientific methods.

Each iteration through the Build-Measure-Learn loop is an experiment which tests one or more hypothesis of the business and, like any endeavour, planning is essential to maintain the focus on the experiment and understand its objectives. The Build-Measure-Learn Iteration Planning Session will answer the following questions:
 

• What hypotheses you are going to validate in this iteration?
• What do you need to learn to validate/invalidate the assumption?
• What metric(s) do you need to collect to validate the learning?
• What do you need to measure in order to validate your learning?
• What do we need to build in order to validate our learning?

 

What you have at the end of the planning session will be a well planned experiment with specific objectives and goal. Everyone on the startup team should take notes during the planning session to document points of views, opinions, debates and any decisions made. What you're trying to capture are the decisions made and any differing opinions which, it may turn out, need to be revisited at a latter date.

 

Select The Riskiest Hypothesis to Test First
 

The startup team should be selecting the riskiest hypotheses from the business plan, marketing plan, or sales plan to test and validate first. This is especially important in the pre-Product/Market Fit phase of the business when the vision of the startup business is unproven and your customers and product are unknown or vaguely known. These risky hypotheses are usually fundamental to the business proposition and if proven untrue, would cause a major change, pivot, in the business.

The team will need to be cautious when selecting hypotheses to test to ensure that any MVP optimization can be unequivocally traced to a specific result. If the team is testing 5 hypotheses but makes one change to the MVP then the cause and effect is generally traceable. If however, the team makes 5 changes to the MVP to test 5 hypotheses then there's no clear cause and effect and the team's learning will be very limited. The startup team should try and keep the number of hypotheses under test in a Build-Measure-Learn iteration as small as practical as a means to limiting the amount of optimization required on the product.

 

Identify learning objectives
 

The team will clearly identify the objectives of this iteration of the Build-Measure-Learn loop. These objectives become the goal of the iteration. For example, assume part of your customer value hypothesis is your product appeals to all age groups. However, during the last Build-Measure-Learn iteration the team learned that the product lacked appeal to the 45 – 65 year old market. During face-to-face interviews with these potential customers, the team determined that the user interface might be overly cluttered for these customers.  This iteration of the Build-Measure-Learn loop will optimize the MVP and test that there's a 30% increase in Activations and a 10% increase in sales to the 45 – 65 market segment. The objectives would then be:

• Improve the user experience by removing unused (greyed out) buttons and controls and increase Activation by customers aged 45 - 65 by 30%
• Improve the user experience by removing unused (greyed out) buttons and controls and increase Revenue from customers aged 45 – 65 by 10%.

It's important that the team be very specific about what they are attempting to do. If the objective was, "Increase Revenue from customers" then how would you know if you were successful? Would $5 be good enough? Would any age group count?

 

Identify metrics
 

The team will select the metrics needing collection for the customer story under test. For the 45 – 65 year old market segment example, the metrics are Activation and Revenue. However, the team may need to also monitor the metrics Acquisition and Retention just to make sure any change to attract one customer segment doesn't cause another customer segment to suffer.

The team needs to be aware that any change made to the product for testing one hypothesis may have desirable and undesirable side effects that the team cannot ignore. The left hand needs to know what the right hand is doing.
 

Identify measures
 

The team needs to determine what and how they measure the customer’s behaviour in order to validate their learning with the next MVP. The measures you select must provide the proof one way or other that the experiment is changing the customer’s behaviour. For example, if you’re trying to increase sales to the aged 45 – 65 market segment, raw sales numbers or revenue will not be the appropriate measure because it’s not clear who is actually buying the product and what their age group is.

The team will need to get some age information from the customers. Assume for the moment that Acquisition was viewing a demo and registering for product download, then part of the registration process might be to have the customer check an age group or give their birth date. Activation might mean the customer installs and runs the product which requires them to login again and get a temporary licence. Revenue again requires the customer to login and give a credit card or paypal account.

Another way to get age information is through face-to-face interviews although asking someone their age might be inappropriate.

The team should also determine how data are collected and presented for evaluation by the startup team.
 

Write MVP user stories
 

Once the team knows what hypotheses are under test, what metrics to use and what customer behaviour to measure, they can focus on the means for conducting the experiment and define the MVP needed for the experiment. For the 45 – 65 market segment example, this would be a software change to the product to improve the customer’s user experience.

However, the team is not limited to having only a software product as MVP. For example, they may use a new marketing promotion or a new sales channel as the MVP. The point is not what the MVP is but the potential for the startup to learn about their customers and business model by using the MVP as the tool for learning.
 

When developing MVP user stories, the team must not lose sight of the startup’s vision and begin making product changes that don’t serve that vision. They must also keep focused on the hypotheses under test and not make any product change that doesn’t support the testing. The MVP is used to prove or disprove specific hypotheses, not to create more uncertainty and complications.
 

The team must also be aware that any change to the product and the subsequent customer behaviour, i.e. cause and effect, must be as unambiguous as possible. The team should only make one change at a time to test the hypotheses. If the team makes multiple changes, it’s harder or impossible to know which change caused what customer behavioural change.

 

Finish Planning

Once the startup team has the MVP user stories, they need to document everything from the planning meeting. This is informal and could be pictures of white boards, annotated scraps of paper, and other notes. What you're trying to capture are the decisions made and any differing opinions which, it may turn out, need to be revisited at a latter date. Save this in a well known location so it can be recovered at some future date if necessary.

If you have any comments or suggestions on planning the activities of the Build-Measure-Learn loop please let me know.

 

Tweet

Startup Process of Vision and Planning

At a recent startup meet up in Sydney I was speaking to a couple of entrepreneurs who had their vision and had an initial MVP but there was a disagreement as to whether they should take the next step and begin showing their MVP to customers.

 

This one scene, probably repeated dozens of times a day all over the world, illustrates clearly the need for some sort of process, a checklist, or best practice, something that can assist the entrepreneur to decide when it's time to move forward.

 

Think of the process of driving a car. What steps do you follow?

1. Unlock car door - Verify: Door can be opened
2. Climb into car - Verify: You’re sitting in driver’s seat
3. Close car door - Verify: Door sounds like it closed properly
4. Put on seat-belt - Verify: Seat-belt is latched and is comfortable
5. Put key into ignition and turn to start car - Verify: Car starts up, engineer sounds OK
6. Put foot on brake and look around - Verify: It seems safe to move car
7. Put car in gear and look around again - Verify: Car lurches slightly in direction you're going and nothing seems to be in the way

and you get the picture. Although we never look at a document or follow any written steps, the procedure exists. It is in the car owner's manual, which is buried in glove box under a bunch of other junk and hasn't seen daylight since the last crisis you had when you needed to know how to add windshield washer fluid. None of us gives driving a car a second thought because it is routine and rarely does anything unexpected happen. But when something unexpected does happen we actually seem to know what to do: call for roadside assistance or call the police or hide the body in the bush or whatever. A startup is only slightly less routine if divided into small enough segments. When something unexpectedly happens to a startup, a startup taking small but quick steps, the number of options available is limited and hopefully, clear. I believe this is the whole point of Eric Ries's book, "The Lean Startup: How Constant Innovation Creates Radically Successful Businesses"; you take small, deliberate steps as quickly as possible, if something unexpected happens, you either persevere or pivot. A startup runs a series of pass/fail experiments on very specific items.

 

Let's get back to my two friends who are debating whether they should start talking to customers (there is a third startup team member who was not present). It struck me that one of them was eager to move forward but the other was fearful of failure. Break down what these entrepreneurs have done or not done so far:

• They've had a vision but is the vision shared among the whole startup team?
• They've developed an initial product, MVP, but are reluctant to go in front of customers with it.
• They have a plan but is the plan written down and agreed to by the team?
• They've gone to a startup meeting because they want insight and help from other entrepreneurs.

It may not be obvious how process can help here but it can, the process of vision and planning.

 

The Process of Vision and Planning

 

How someone has a flash of brilliance is yet unknown to humanity and is not subject to a predictable process. However, what happens afterward is. I've labelled these Day 1, Day 2, ... as a matter of convenience rather than dictating the speed which these steps happen. Leave it to the startup team to determine the length of each step but remember, speed is important.

 

Day 1

The vision is the problem statement that the visionary thinks he or she can solve and is a problem that needs to be solved. Rather than focusing on a specific solution, the vision statement captures the problem a startup will be trying to solve. A vision statement may be as follows:

  

Help wireless technology users track their usage.

 

You might be tempted to solve this problem with a mobile app, but if the team later learns through empirical testing and validated learning that most customers don’t want to use a mobile app, you can easily pivot because your vision is solution independent. By framing the product vision as a specific problem rather than a specific solution, you’ll have the freedom to create different types of solutions. This is important as it's likely your first solution won’t be attractive to a broad customer base. By having the freedom to explore multiple solutions this won't be a great concern. Your vision statement gives you the freedom to keep trying rather than hitting a dead end.

 

The visionary may contact a few people to see if the problem is something that they would like to see solved. In Eric Ries's book, Lean Startup, he has a few examples of this with one visionary calling people randomly, one hanging out in a grocery store talking to shoppers, and another who used a YouTube video to demonstrate a solution to a problem. None of these probably took more than a day to complete and although these activities do not validate the vision, they do give the visionary enough feedback to know whether the vision is worth pursuing.

 

So what do you do if no one has the problem you envision to solve? The real answer lies within the visionary and startup team. The visionary and startup team’s hunches, insights, wisdom, intuition, and world experiences must be balanced against customers not having or not knowing they have a problem. However, the startup team must test this hypothesis very early on as part of their customer discovery efforts.

 

Day 2

Assemble a small team of 3-5 people who share the vision. The initial core team is small to facilitate communications, help ensure that the vision is commonly understood, and to foster understanding of the customer’s problem as it relates to the startup’s vision. The point is to have enough people in the startup to objectively learn if the vision is valid but keep the initial costs low and limit waste. But the most crucial thing is that the team shares the vision. I think this is a most crucial requirement and should not be compromised.

 

Everyone on the Team Shares The Vision

 

If the visionary is recruiting a team, he or she should ensure that all potential team members can and do share the vision. Why is this so important? It's important that everyone is 100% committed to the vision so that 100% of their energies are focused on making the vision a reality. Think about a time when you and another person were driving somewhere and disagreed on what route to take. It may end up as an argument or at least one person being moodily quiet while the other defends their point of view. All that's happened is the two do not equally share the vision of what route to take and it has become divisive. A startup will almost certainly fail if the founders are at odds with each other on the first day and cannot agree on the vision - the problem the startup hopes to solve. If one of your team does not appear to share the vision, then ask Pete Best, or whatever their name is, to leave. You can try and persuade them to share your vision but if you can't convince them after an hour, they'll probably never be convinced.

 

Everyone On The Team Is Cross-Functional

 

The startup team is cross-functional. Everyone on the team is expected to talk with customers. Everyone on the team is expected to understand the customer’s problem. Everyone on the team is expected to understand the solution. And everyone on the team is expected to learn throughout the customer discovery and validation process. Having said this, a startup team is not expected to have equal knowledge on every aspect of the business or solution but rather have the capability to contribute and understand all aspects of the startup’s endeavours. It is important that all team members can step outside their primary roles and  contribute to any part of the startup’s business.

 

The skills a startup team will need are:

1. Designer – Designing what may turn out to be several different solutions (MVP) that will test the business assumptions.
2. Developer – Technical ability to create the solution (MVP) in whatever form it takes.
3. Marketing/Sales – Knows how to talk to the customer. Can identify early adopters and understands product positioning, pricing, and promotion.
4. Visionary – Creative genius, curious and risk taker.

One startup team member should also be the startup evangelist; keeping all other members focused on understanding who their customers are, the customers’ problem and working toward a solution to solve that problem.

 

Each startup team member should agree how much time and money they can give to the business startup. Time is self-explanatory but money, or the lack of, can break a startup team very early on. If there's office equipment, hardware/software necessary to build an MVP, travel expenses to visit customers, cost of outside consultants, or any of a dozen other minor or major expenses, the team might need a war chest from which draw cash from.

 

Day 3

Once the core team is assembled, create a business plan using the Business Model Canvas as defined in Alexander Osterwalder and Yves Pigneur's book, "Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers". At this stage, only the items Customer Segments, Value Proposition, and Channels are needed although you can complete the cost and revenue items if you know them or think you might want to feed your family every day (this is where the war chest comes in handy). During these early days, you'll be more focused on discovering who your customers are so don't count on receiving too much in the way of revenue from them.

• Customer Segments-identify who your customers are, people who have the problem you've identified in your vision and will be served by your eventual solution.
• Value Proposition-this is your solution to the problem your customers have. Under the umbrella of your vision, there may more than one problem and solution you are trying to solve.
• Channels-How you plan to reach your Customer Segments to deliver a Value Proposition. This may be different for each Customer Segment you have identified.

It may take a day or two for the core startup team to agree on the business plan. As the team talks through the business plan, the team should thoroughly document this discussion as any thoughts tabled may in fact prove to be significant later.

 

Having a small team will help with reaching agreement on the business plan as a first step toward a successful and sustainable business. The team doesn't necessarily agree that everything in the business plan is absolutely right, in fact the business plan here is not much more than a list of unproven hypotheses and assumptions. What the team agrees to is that the business plan forms the basis from which the team can begin experiments to discover who their customers are, what problem(s) these customers have, and if the team can devise a solution that resonates with these customers.

 

Variations of the business model canvas or lean canvas, can be found in Steve Blank and Bob Dorf's book, "The Startup Owner's Manual: The Step-By-Step Guide for Building a Great Company" and in Ash Maurya's book, "Running Lean: Iterate from Plan A to a Plan That Works".

 

Now What?

At this stage I believe you have completed the vision and planning stage and are now ready to enter the experimentation stage of the startup. But what about my entrepreneur friends I spoke of earlier who seemed to be at this junction; why do they hesitate?

 

I got a strong impression the three entrepreneurs shared the vision of the startup but I think their issue is that the developer had built a solution to the problem before they actually spoke to any customers and the fear of rejection weighs heavy on him. The developer kept asking how they could validate  that customers existed for the solution they had. He wanted to know that customers accepted the solution before telling them what the solution was or showing it to them.

 

The developer would have been less worried about acceptance of the solution if on Day 1 some verification was made that at least a couple people had the problem and wanted a solution as the developer envisioned. This is similar to Eric Ries's problem in his IMVU company where he built a solution to a problem before testing the Value Proposition on customers. If you recall, Eric Ries was very reluctant to throw away his code and stubbornly stuck to his solution even if it meant "firing customers" who didn't accept his solution. A startup is most certainly going to build some MVP before knowing if any customers want it but having some reassurances by verifying with customers on Day 1 might have given a developer the confidence necessary to begin startup experimentation.

 

I also think, although this is an assumption on my part, that the team might have had a plan but it wasn't written down anywhere and the team of three did not agree to it, at least not in so many words. The entrepreneurs may not have remembered that the business plan is neither fact nor fiction but is mostly a list of un-validated hypotheses and assumptions. A written plan has a much stronger impact that one that is verbal. It's easier to ask questions about written words, hypotheses, and assumptions. Writing the business plan down and agreeing to it can help reinforce the idea that the plan is a point of departure, that the one thing the team knows and agrees to is the plan they see on Day 3 is not complete and not correct. The team knows the plan will change as they learn more about their customers, their customers' problems, and the differing solutions the team might offer. By the team agreeing to the plan, they also agree to the uncertainty of that plan. The developer might have still felt anxious but the other team members and the developer, having agreed to the plan, have now accepted joint ownership of the plan, MVP and the risks involved with both the plan and MVP.

 

By the way, my advice to the entrepreneurs was to begin conducting experiments to validate their plan and showing the MVP to customers-but don't be reckless about it. Besides, what did they have to lose?

 

References

These are books I would recommend you buy and read:

• Ries, Eric. "The Lean Startup: How Constant Innovation Creates Radically Successful Businesses"
• Maurya, Ash. "Running Lean: Iterate from Plan A to a Plan That Works"
• Blank, Steve; Dorf, Bob. "The Startup Owner's Manual: The Step-by-Step Guide for Building a Great Company"

 

 

Tweet

Lean Startup Process: Starting the Blog

Hi Everyone,

 

I'm planning on documenting what Best Practices and process a business startup should follow, specifically startups using Eric Ries's Lean Startup framework. These best practices are meant to be a day-to-day guide for startups to help them stay on track by asking itself the right questions at the right time during their journey from initial vision to product/market fit and beyond. Along with hearing stories of harrowing miss-steps and jubilant successes, I am looking to get what specific and deliberate steps a startup took to either move toward success or move toward gloom. Underlying these steps are the indications and indicators, like tea leaves, which may have helped predict the startup’s future progress and success. If you have a story about your startup, I hope you can share it to help other startups.

 

In the mean time, I'll be publishing whatever information I've gathered or received here for everyone's benefit.

 

Thanks for your time and hope your visits here are most productive, Bob

Tweet