The Build-Measure-Learn loop is the heart of Lean Startup and every iteration through the loop requires some degree of planning. Build-Measure-Learn iteration planning is similar to Agile iteration planning where the startup team plans the activities and identifies the expected results for the next iteration of the Build-Measure-Learn loop.
Build-Measure-Learn Iteration Planning Session
The purpose of the Build-Measure-Learn iteration is to test some unknown aspect of the business. This might be an experiment to learn about customers, the product, sales, marketing, support, packaging, or something else that the business feels will increase the value of the product or increase the growth of the business. Whether the business is in pre- or post-Product/Market Fit, the Build-Measure-Learn iteration serves the same purpose: learning to improve and grow the business using scientific methods.
Each iteration through the Build-Measure-Learn loop is an experiment which tests one or more hypothesis of the business and, like any endeavour, planning is essential to maintain the focus on the experiment and understand its objectives. The Build-Measure-Learn Iteration Planning Session will answer the following questions:
Each iteration through the Build-Measure-Learn loop is an experiment which tests one or more hypothesis of the business and, like any endeavour, planning is essential to maintain the focus on the experiment and understand its objectives. The Build-Measure-Learn Iteration Planning Session will answer the following questions:
- What hypotheses you are going to validate in this iteration?
- What do you need to learn to validate/invalidate the assumption?
- What metric(s) do you need to collect to validate the learning?
- What do you need to measure in order to validate your learning?
- What do we need to build in order to validate our learning?
Select The Riskiest Hypothesis to Test First
The startup team should be selecting the riskiest hypotheses from the business plan, marketing plan, or sales plan to test and validate first. This is especially important in the pre-Product/Market Fit phase of the business when the vision of the startup business is unproven and your customers and product are unknown or vaguely known. These risky hypotheses are usually fundamental to the business proposition and if proven untrue, would cause a major change, pivot, in the business.
The team will need to be cautious when selecting hypotheses to test to ensure that any MVP optimization can be unequivocally traced to a specific result. If the team is testing 5 hypotheses but makes one change to the MVP then the cause and effect is generally traceable. If however, the team makes 5 changes to the MVP to test 5 hypotheses then there's no clear cause and effect and the team's learning will be very limited. The startup team should try and keep the number of hypotheses under test in a Build-Measure-Learn iteration as small as practical as a means to limiting the amount of optimization required on the product.
The team will need to be cautious when selecting hypotheses to test to ensure that any MVP optimization can be unequivocally traced to a specific result. If the team is testing 5 hypotheses but makes one change to the MVP then the cause and effect is generally traceable. If however, the team makes 5 changes to the MVP to test 5 hypotheses then there's no clear cause and effect and the team's learning will be very limited. The startup team should try and keep the number of hypotheses under test in a Build-Measure-Learn iteration as small as practical as a means to limiting the amount of optimization required on the product.
Identify learning objectives
The team will clearly identify the objectives of this iteration of the Build-Measure-Learn loop. These objectives become the goal of the iteration. For example, assume part of your customer value hypothesis is your product appeals to all age groups. However, during the last Build-Measure-Learn iteration the team learned that the product lacked appeal to the 45 – 65 year old market. During face-to-face interviews with these potential customers, the team determined that the user interface might be overly cluttered for these customers. This iteration of the Build-Measure-Learn loop will optimize the MVP and test that there's a 30% increase in Activations and a 10% increase in sales to the 45 – 65 market segment. The objectives would then be:
- Improve the user experience by removing unused (greyed out) buttons and controls and increase Activation by customers aged 45 - 65 by 30%
- Improve the user experience by removing unused (greyed out) buttons and controls and increase Revenue from customers aged 45 – 65 by 10%.
Identify metrics
The team will select the metrics needing collection for the customer story under test. For the 45 – 65 year old market segment example, the metrics are Activation and Revenue. However, the team may need to also monitor the metrics Acquisition and Retention just to make sure any change to attract one customer segment doesn't cause another customer segment to suffer.
The team needs to be aware that any change made to the product for testing one hypothesis may have desirable and undesirable side effects that the team cannot ignore. The left hand needs to know what the right hand is doing.
The team needs to be aware that any change made to the product for testing one hypothesis may have desirable and undesirable side effects that the team cannot ignore. The left hand needs to know what the right hand is doing.
Identify measures
The team needs to determine what and how they measure the customer’s behaviour in order to validate their learning with the next MVP. The measures you select must provide the proof one way or other that the experiment is changing the customer’s behaviour. For example, if you’re trying to increase sales to the aged 45 – 65 market segment, raw sales numbers or revenue will not be the appropriate measure because it’s not clear who is actually buying the product and what their age group is.
The team will need to get some age information from the customers. Assume for the moment that Acquisition was viewing a demo and registering for product download, then part of the registration process might be to have the customer check an age group or give their birth date. Activation might mean the customer installs and runs the product which requires them to login again and get a temporary licence. Revenue again requires the customer to login and give a credit card or paypal account.
Another way to get age information is through face-to-face interviews although asking someone their age might be inappropriate.
The team should also determine how data are collected and presented for evaluation by the startup team.
The team will need to get some age information from the customers. Assume for the moment that Acquisition was viewing a demo and registering for product download, then part of the registration process might be to have the customer check an age group or give their birth date. Activation might mean the customer installs and runs the product which requires them to login again and get a temporary licence. Revenue again requires the customer to login and give a credit card or paypal account.
Another way to get age information is through face-to-face interviews although asking someone their age might be inappropriate.
The team should also determine how data are collected and presented for evaluation by the startup team.
Write MVP user stories
Once the team knows what hypotheses are under test, what metrics to use and what customer behaviour to measure, they can focus on the means for conducting the experiment and define the MVP needed for the experiment. For the 45 – 65 market segment example, this would be a software change to the product to improve the customer’s user experience.
However, the team is not limited to having only a software product as MVP. For example, they may use a new marketing promotion or a new sales channel as the MVP. The point is not what the MVP is but the potential for the startup to learn about their customers and business model by using the MVP as the tool for learning.
However, the team is not limited to having only a software product as MVP. For example, they may use a new marketing promotion or a new sales channel as the MVP. The point is not what the MVP is but the potential for the startup to learn about their customers and business model by using the MVP as the tool for learning.
When developing MVP user stories, the team must not lose sight of the startup’s vision and begin making product changes that don’t serve that vision. They must also keep focused on the hypotheses under test and not make any product change that doesn’t support the testing. The MVP is used to prove or disprove specific hypotheses, not to create more uncertainty and complications.
The team must also be aware that any change to the product and the subsequent customer behaviour, i.e. cause and effect, must be as unambiguous as possible. The team should only make one change at a time to test the hypotheses. If the team makes multiple changes, it’s harder or impossible to know which change caused what customer behavioural change.
Finish Planning
Once the startup team has the MVP user stories, they need to document everything from the planning meeting. This is informal and could be pictures of white boards, annotated scraps of paper, and other notes. What you're trying to capture are the decisions made and any differing opinions which, it may turn out, need to be revisited at a latter date. Save this in a well known location so it can be recovered at some future date if necessary.
If you have any comments or suggestions on planning the activities of the Build-Measure-Learn loop please let me know.
No comments:
Post a Comment