A: Very often the equations are not known; often, they are unknowable. In those cases, it is fine to just test similar physical designs to capture their behavior. Or if you don’t have anything to test, or it is too expensive to test, then you can rely on simulations or other computational models to generate “response surfaces” that you can design to. Lacking such models, you can resort to “rules of thumb” and the other rough estimates that your engineers are already relying upon to make the decisions they make. The key is to make visible whatever knowledge it is that your engineers are currently using, such that such knowledge can be understood, challenged, and improved over time.

A: The beauty of Set-Based Analysis is that it can accommodate uncertain data. You can use simple “rules of thumb” and “gross estimates”, capturing them as visible knowledge that you can use to limit the sets being considered now, and then as you gradually learn more, then you can further converge those sets, eliminating the weak as you determine what is weak. (TCC’s Success Assured™ software will even let you literally just draw the limit curves and response surfaces, relying entirely on what’s in your experts’ heads, when there is no data available. And then adjust those rough curves/surfaces to hit the few data points that you might have from past products or competitor products.)

A: Maybe. It depends heavily on how close to the limits your prior product design was. And since point-based methods don’t expose that (unless you exceed one of those limits), you generally have no idea how close you are to those limits, and whether your revised product design is going to be moving towards those limits. However, over time, as our products get more optimized and more complex, the likelihood of small changes resulting in exceeding one of the many limits grows exponentially. So, your particular project might get lucky and work out just fine with traditional point-based practices; but the majority of today’s projects end up being late and over-budget, because most don’t get so lucky. And the more optimized and more complex products get, being so lucky will get rarer and rarer.

A: Absolutely! Until independent researchers produce comprehensive hard data showing the benefits of set-based practices, many development organizations will be reluctant to depart from their traditional point-based practices, no matter how big the potential benefits appear to be. These Set-Based Games can be used as the foundation for formal research studies where different groups of engineers perform the design work either point-based or set-based and the impacts are captured and analyzed. Further, those can be extended into longer projects, simulating the typical shifting of requirements over time, or the introduction of new knowledge, or similar, and then study the impact on the projects being conducted point-based vs. set-based. If you are interested in conducting and publishing such research, we are happy to help however we can.