Day Two – What are you aiming for?
Benchmarks are vital in any round of product concept testing. What we’re doing with benchmarks is effectively creating our success rate, our pass / fail parameters.
Imagine for a moment that you’re an archer… you don’t just shoot arrows in a vaguely random forward direction – there is a defined target that you are aiming for. This is where benchmarks come in. Running tests on product concepts without using benchmarks each time is like shooting arrows without a target. The product concepts may feel right, and the scores may look good, but without benchmarks you just don’t know if they’re going to hit the target.
So we need to work up some benchmarks, and include them in the testing we’ll be doing imminently. But what do we choose, and how do we present them?
A good benchmark would be an existing product that is on sale in the category currently, whose performance level was around the minimal acceptable level that a new product would need to achieve to justify its place on shelf. Don’t choose the best selling products, as that expectation is unrealistic for a new product, and you’re setting your bar too high. Equally don’t make life easy for yourself and choose the lowest performing products in the category. Average, minimum expectation level is what we’re looking for here. And ideally don’t just choose one benchmark, choose quite a few. This means you can take the average of how they perform in the testing and use that average. It makes it more statistically robust than just using one in isolation.
Action: Develop your benchmarks.
Create a product title, around 50 characters, and then add a product description of around 150 characters. You’ll see us talk about consistency a lot within Agile – presenting products & benchmarks in a consistent way either with descriptions or images is crucial during every round of testing.
The benchmarks will be put through the testing process in a couple of days time in exactly the same way as the product concepts will be. The scores they get, when combined together as an average across all benchmarks, will form your acceptability or pass / fail rate. This makes it so much easier to make empirical decisions about which product concepts to take forward from each round of testing.
Tomorrow on Day Three we’re going to discuss images – for both product concepts and benchmarks.