Are Industry Surveys Just “Alternative Facts?” How to Use The Information They Provide.

Industry surveys are everywhere. The term “alternative facts” has recently come under scrutiny and provides good insight into much of the value of industry surveys. To quote Mary Norris’ recent article in The New Yorker, “They do not have actual existence. Alternative facts are delusions.” The same holds true for many of the claims made in the ubiquitous industry survey. That being said, industry surveys still have some use for executives and managers.

The industry analyst community makes a pretty good living generating industry surveys, for example, the Gartner Supply Chain Top 25. Unfortunately, very many of these surveys are completely unscientific. They do not pose a hypothesis and then rigorously test the hypothesis to determine if the hypothesis can be rejected or not–anybody remember the scientific method basics from high school? Very many of these surveys use no statistical analysis to determine validity of the findings. There is no determination of causality. Saying “80% of ‘best performing’ companies use ERP systems” provides absolutely no value in predicting whether acquiring an ERP system will vault a company into the ranks of best performers. Depending on the pseudo-statistics typical in industry surveys is essentially a “management by folklore” approach heavily dependent on luck for success. Predictable and profitable performance depends heavily on understanding and managing the practical science governing relationships between demand, cost, inventory, capacity, variability and response time.

The infographic shown provides a typical industry survey example. See the full size version at the Manufacturing.net website. The survey addresses the following question, “What do manufacturers say will deliver growth?” To their credit, the surveying group only claims to be reporting what manufacturers say drives growth, not what actually drives growth. The infographic shows the “Path to Growth” starting with “Good planning” (60%), then “Hard work and determination”(48%) and on down the path till the last item is “agree that an effective and integrated IT infrastructure is essential for business performance” (82%). Note there is no qualification of business performance, i.e. profitable performance or dysfunctional performance, just that IT is essential for business performance. What did companies do before IT? Was there no business performance?  The numbers in parentheses presumably represent the percent of respondents that selected that response.

An even bigger whopper of an alternative fact is a statement made in the conclusion of the report from which the infographic is taken. The report states, “The common theme across all these findings is that technology will make the difference between success and failure for manufacturers.” By technology, the report is referring to information technology. At FPI we have personally been witness to many industry refutations of that statement. Our experience is that many manufacturers, if not most, succeed or fail not because of technology but in spite of it. IT vendors want to sell you IT. Many of them have no idea of the underlying behavior of the systems they are trying to automate. See pp. 200-202 in Factory Physics for Managers for more explanation.

Alternative facts are stated regularly in industry surveys. If there is any value in most industry surveys, it is in informing industry of the latest trends–often those with heavy marketing budgets–and providing a tabulation of the concerns of executives and managers. Our experience is that most companies can greatly improve performance on inventory, cash, profit and customer service by applying a practical scientific approach to their business using their existing information technology and in-house continuous improvement resources. The correct application of operations science is a fundamental driver of predictable and profitable performance. Executives and managers should beware of the widespread use of alternative facts in industry surveys.

    Comments are closed