Testing has a very different evolution comparing testing of software with testing of hardware.

Testing has several purposes: customer feedback, risk mitigation, learning, validation, etc. It’s also a great source for innovations. Why I ask myself if the best innovators are also the best on exploration how do they create more options through testing?

Test for learning and to explore

Innovators have a growth mindset; they test for learning and to explore a world of opportunities. In contrast to a traditional development approach with a fixed mindset, which mainly test for defects when the final product is ready. While most companies want to increase both their innovation capability and pace, many times, the trend is in the opposite direction. Could it be that the very process ‘improvements’ they develop, work against the desire to improve innovation and speed? Let’s look deeper into trends in testing that may help explain this disconnect.

Over time companies have added many mandatory analyses to their project plans. And at the same time tried to reduce expensive physical testing, with only one single flawless production validation as a haunting end goal.  In other areas, such as software development, testing protocols have moved in the opposite direction. Testing of software has moved from a few back-end validations of final code to instant front-end tests of all added and changed code. At the same time testing of hardware to a large extent has moved in the opposite direction. The latter is currently evolving towards a single final back-end production validation of the final product. With this evolution, the test equipment that is kept is only capable of system testing. Subsystem test equipment for front-end testing is being replaced by computer-aided engineering and simulations. Testing of hardware that is seen as high cost and long lead time, risks becoming even more expensive and can delay a project when unknowns emerge during the first actual tests that occur late in the project during validation.

There is a non-linear relation between Scope versus Reward or Harm from testing. A metaphor explains the non-linearity. Relying on back-end system testing, is in comparison like falling from ten meters. The later actually seems to be the cutoff point for death from free fall. If you fall twice from five meters you survive, but probably are severely harmed.  Even better, “falling from one meter (3.3ft) ten times cause you less harm, than falling from ten meters (33ft) one time.” And the harm falling from ten meters is more than ten times the harm falling from one meter.

Comparison of multiple small tests vs a single large system test

We do better by thinking small

In addition, we do better by thinking small. One huge problem can be defined as a network of smaller problems. Looking with the Innovator’s eyes, we can work with small wins and survive small failures. We can identify manageable opportunities with smaller scope and shape a success. Every small win brings more options, every small failure presents new learnings. The many small wins can have explosive payoffs, with unknown upper bound.

To ensure customer feedback is built-in, optimize learning, growth, and subsequent success we should apply a strategy of small, cheap and fast trials. Using this strategy, innovators create optionality and wins, by falling and getting up multiple times. Small, desirable failures, push the boundaries allowing us to learn the outer limits. Falling, at multiple small trials, is one of the best sources for innovation. With a growth mind-set, we expand our thinking with “AND”, we can “ratchet up” on side findings – “mutations”. Findings who come at no cost and are retained only if they are an improvement, adding further optionality for future customer offerings. Companies recognized for their innovation speed, gains from falling are an important part of their strategy. Because their falls are many, but small and cheap, they not only stay in the game but often lead the innovation race.

Do as the best Innovators – keep it simple, wrap up the simplest rapid hardware prototypes and test single or a few parameters at the time.

Christer Lundh

To summarize, in the effort to take advantage of innovations you should focus on small and rapid tests to improve the payoff function of tests / trials. It is easier to make small trials to learn what doesn’t work, than to figure out exactly what knowledge we are looking for. We are usually better at doing (testing), than we are at thinking (modeling). With lower costs and cycle times per unit of trial, we reduce the downside of testing. Because we reduced cost and time per test, we have the flexibility to multiply the number of trials, which allows us to minimize the probability of missing with the potential to maximize profits should one have an unexpected early success. A single system validation on the final product is a risky approach with a larger expected downside, than a portfolio of small subsystem trials.

Do as the best Innovators – keep it simple, wrap up the simplest rapid hardware prototypes and test single or a few parameters at the time. Build-in prompt customer feedback by falling multiple times from “one meter,” rather than risk a catastrophic 10-meter fall – embrace the better practice of “Test, Test, Test.”


Christer Lundh

Christer Lundh

president of AUFERO

AUFERO makes organizations more profitable by improving product development and innovation. Christer is a board member at LPPDE.org. A passionate leader co-creating flow of new products and innovations together with clients.

Christer Lundh is speaking at LPPDE Europe 2021, May 19-21. 

He will share his insights on the benefits of increasing testing early in the development and how it can lead to more learning and profit while reducing overall project risk, see link below for registration.

Join the Lean Innovation Virtual Knowledge Exchange!