A test and learn approach, in contrast, lets you try, learn, and perfect concepts on a smaller scale, with less risk, to gather the facts needed to determine which ideas offer the most promise.

We’ve seen this approach work as an effective way of jump starting innovation at some organizations or encouraging evolution at others. We think that the notion of test and learn—which has traditionally been viewed as a tactical tool—should be elevated to a strategic framework for the C-suite. Whether you view this expansion as a dynamic version of scenario planning or something bigger—even a way to develop new ideas and fast market responses that challenge the most “knowledgeable” of industry thinkers and investors—test and learn as a strategic application holds great, but largely has untapped potential.

A Better Way for the C-Suite to Allocate Resources

Over time, test and learn has expanded from being just the domain of larger companies to becoming more democratic, with businesses in the $5 billion dollar range and under regularly applying it to try out things like store format, product, pricing, staffing levels, technology, and, of course, advertising and promotional strategies.

At Stax, we believe that with the right setup, test and learn can and should be used to a far greater extent at higher levels of the management hierarchy.

With the expansion of big data and more companies taking a more analytical approach to decision making, the C-suite now has even more of an opportunity to apply test and learn to new mission-critical, organization-wide strategies. While the marketing guys are testing granular tactics like couponing, the C-suite can be using test and learn to address broader strategic issues to help guide capital investments or budgeting decisions.

For example, which is likely to get a higher ROI or ROE: a new media buy or a new plant? Cutting prices or adding new distribution channels? Being able to test which allocation of capital will deliver the highest return makes your capital-allocation decisions more informed and effective.

Similarly, investors such as private equity firms developing post-deal-closing 90-day plans may not focus solely on the “three big levers” to grow the company. They may, in fact, find it better to test multiple levers that can work collectively to move a portfolio company’s business forward and keep it on a steady, profitable growth plan. Using test and learn for these types of strategic purposes is new and may be subject to some internal resistance. But the benefits of a smoother strategy implementation resulting from testing in advance, should quickly win over the naysayers.

The way to think about test and learn is this: It doesn’t matter if you’re right the first time, as long as you get it right in the end—and use that right answer to make right-sized efforts and investments. As long as you start small, you can contain the costs of ideas that do not work, while building on positive results.

Say, hypothetically, an idea has a range of potential outcomes from a 1% sales increase to a 2% decrease. Because of this range of possible outcomes, there is real risk involved in moving forward with it system wide. If, instead, you try the idea first on just 0.5% of the customer base, the range of potential impact drops to between 0.0005% and -0.001%—which is miniscule. If it’s a failure, you walk away and learn from the experience. If it works, you roll it out more broadly. If it produces some improvement but falls short of targets, you can adjust the idea and re-test it. And, with fast cycles of iteration, you can continuously test, learn, and improve the idea so when you launch it, you have dramatically improved the probability of success.

Using Test and Learn for Customer Segmentation

You are well aware that not all customers have the same needs, nor does your business face the same competitors across various categories and customer segments. Great operating and marketing teams need to have more products, more ways to merchandise, and more configurations than in the past. Management teams may find that certain growth strategies are more effective in some businesses than others. Test and learn provides an invaluable way to sort out potential improvements for specific segments before moving forward full steam.

Getting the Analytics of Test and Learn Right

Sample sizes in the test and learn approach may be small, but the science around it is not. In fact, for test and learn results to have legitimate meaning, the smallness of the samples makes it even more important to make sure that the analysis is quantitatively “pure.” The small test groups must be carefully selected to provide a valid representation of the broader customer base or specific target market. The various groups need to be comparable among each other when setting up what is, effectively, a competition between alternative ideas.

The math to do all this correctly is complex. You don’t personally need to understand it, but it’s important that you, or a designee close to your business strategy, work with someone who does understand the analytical components to ensure that the findings are actionable.

There also needs to be discipline around the analytic approach being applied: Is it a straightforward comparison of A vs. B, looking to see which works better? Or is it a multivariate analysis looking at an intricate issue on several dimensions simultaneously, such as a combination of performance attributes like revenue and profitability along with functional attributes such as store size, location, and proximity to a competitor location?

Stax is constantly working with clients to help them identify profitable opportunities to grow their companies and create value, operationally and investment-wise. Actionable and data-driven results are our calling card. We see great opportunity to apply big data and seemingly tactical methods to what has classically been the ivory tower of “industry knowledge” and high-level strategic thinking.

By bringing more facts, analysis, and market agility to strategic decision making, the knowledgeable can become even better, faster, and more successful in leading organizations in this uncertain, dynamic world.