wisdom of crowd


In my quest to write hypothetical case studies on companies & organizations that need Prediction Markets, I’ve admittedly missed many easy opportunities. Just a quick mental scan of this summer’s headlines produced several ripe candidates: The Yahoo!-Microsoft Deal, post-bankruptcy General Motors, the “closing” of Guantanamo Bay, the Cash for Clunkers program, etc. All are business or political endeavors whose fate is (or was) unknown, but whose future could have been predicted, potentially, by the proverbial crowd. If only the crowd had a Prediction Market, that is.

A Prediction Market is a crowd-sourcing tool that organizations can use for improved decision intelligence. Employees play the market … executives get better information. These internal markets can be (and are being) used for forecasting, new product development, capital investments, and increasingly, project management.

But instead of jumping on these summer headlines and blogging up a storm, I got caught up with my own project management woes. Too many things to do, not enough time, yadda yadda.

In my case, there’s no real cost to delaying my next blog post. But in the case of major initiatives in Corporate America or our government, there is plenty at stake. The costs of missed deadlines, inaccurate sales forecasts, budget shortfalls, marketing flops, rejected legislation, or failed mergers are significant, and scary. Which is why the relatively simple and inexpensive prediction market solution is so compelling.

As I’ve written before, Prediction Markets can’t solve everything. But they can provide information that decision makers just can’t get anywhere else. They can ask Yahoo! and MSFT employees for individual prognoses on the partnership, then aggregate it into actionable data. They can ask GM dealers across the country how many customers will return cars under the 60-day guarantee, ensuring the benefits exceed the costs. They can ask government officials from disparate branches the likelihood of Gitmo closing on schedule, uncovering hurdles and loopholes.

When the answers to these big questions can’t be found among the usual experts or with the usual tools … wise companies are beginning to trust the wisdom of the crowd. They’re experimenting with prediction markets, wiki’s, open innovation models, and the like. They’re trusting the insights and ideas of their various stakeholders, not just their executives.

But many companies aren’t. Many companies look at Prediction Markets and similar cutting-edge tools as unnecessary costs without guaranteed returns. This sounds interesting, but what’s the ROI? That, the quintessential business question, can’t be shunned. But it only takes a brief contemplation of the costs associated with big business blunders to entertain a new form of the question:

What is the cost of not doing?

Advertisements


James McNerney, Chairman/CEO

Boeing Corporation

Dear Mr. McNerney,

Do you sometimes feel like the last to know? Like people won’t tell you what’s really going on? Like transparency is demanded of you, but not always provided to you?

This week’s headlines give me an inkling that you might. As I read the news about the latest unexpected delay of your Dreamliner jet, I tried to put myself in your shoes. If only you had known this would happen … earlier. Perhaps you could have avoided the bad press, the analysts’ chastisement, and the customers’ ire. Perhaps you and your team could have even avoided the delay all together.

Such information deprivation, if the WSJ’s assumption is true, seems unfitting for a CEO. But then again, you’re not new to this gig. You know this is a common ailment in the corporate upper-echelon. Communication woes can be the byproduct of the very systems and norms that make big companies work

But like many textbook management problems, this one has a solution set. As the cliché goes, the answer is often in the room. Or in this case, allow me to suggest that the answer is in the crowd.

Respectfully yours,

-A Prediction Markets evangelist


I contemplated starting a Prediction Markets blog quite a while ago. But like the beleaguered Boeing 787, it never seemed to get off the ground. Turns out, all I needed was a hook. Unfortunately for Mr. McNerney, his current predicament fit the bill.

Actually, the real catalyst was my former boss and Prediction Markets partner at Best Buy Co., Jeff Severts. During the 3+ years that we tinkered, tested, and touted internal corporate markets together, Jeff would often cite Boeing as the perfect candidate for a Prediction Markets case study. Following the Dreamliner saga in the news, Jeff was quick to recognize that Jim McNerney likely wasn’t getting the information he needed to make effective decisions, announcements, or promises.

At Best Buy, we (and our senior leaders) often felt the same pain. That certainly didn’t make us unique. What was unique was the candor with which our executives acknowledged this challenge. This organizational self-awareness, combined with our corporate culture of valuing all employee insights, made us fertile ground for incubating a Prediction Market. So we did.

But this blog is not about Best Buy.

This blog is a “WHAT IF” exercise … taking headlines from the world of business, government, social enterprise or economics … and imagining how things might be different if Prediction Markets and other Collective Intelligence mechanisms were widely adopted.

Prediction Markets are fundamentally about tapping the wisdom of the crowd and unearthing information that is otherwise hidden. In these speculative markets, participants trade assets representing unknown future outcomes, essentially ‘betting’ on events they deem most likely. Rewarded for accuracy, traders are motivated to act on information rather than pure speculation. As the market dynamically aggregates all this information, asset prices can be interpreted as probabilities or expected values. Such collective forecasts have often proven to be more accurate than traditional predictive indicators. This has fueled their growing visibility and popularity in recent years. More on PM’s can be found here.

Used within organizations, Prediction Markets are uniquely able to correct information flow inefficiencies caused by bureaucracy, time, human nature, politics and hierarchy.

And so, back to Boeing.

Assuming that at least some, if not all, of these inefficiencies are at play within Boeing, let’s imagine how Prediction Markets could potentially change the course of events in this developing story.

WHAT IF … Boeing used Prediction Markets in its project management toolkit?

Problem:

  • Six major delays over a 6-year period for Boeing’s highly anticipated Dreamliner 787 Jet.
  • Problem exacerbated by several executive statements (some as recent as last week) promising, or otherwise suggesting that flight testing was on schedule and imminent.
  • Latest announcement, along with lack of a revised ETA, sends Boeing’s stock down 12%.
  • Meanwhile, its leadership is taking heat from the press and Wall Street regarding its ability to effectively manage and communicate.
  • Delays have been blamed on factors such as outsourcing, parts shortages, vendor miscommunications, design flaws, labor strikes, and installation errors.
  • With this series of missed delivery dates, Boeing loses credibility with key stakeholders and stands to lose significant profitability.

Hypothesis: Project management of the 787 lacks a sufficient information flow mechanism. This results in delays and/or errors in what’s being communicated up the chain of command, and subsequently to external stakeholders.

Proposal: Boeing should launch an enterprise prediction market, with specific contracts focused on delivery metrics of the 787. The following is a high level summary:

What type of prediction market?

The Boeing case is a good application for project management-based markets – which help monitor whether a project will be on time, on budget, and/or to spec. Boeing could translate any number of questions concerning the project’s health into contracts, such as: Will a particular milestone be reached? Will this budget be exceeded? Will we pass a certain inspection? As traders take positions in these stocks, the magnitude of their confidence or skepticism will influence how much they wager. Once the future event is eventually measured, they will profit or lose accordingly.

Who would trade in this market?

As with any enterprise market, management can decide whom to invite. After that, it’s all voluntary. On the most conservative side, Boeing could limit trading to certain divisions, departments or ranks. A more typical approach would be to include all employees. And at the most ambitious end, McNerney and team could incorporate supply chain partners or even customers into its trader base – acknowledging that in today’s globally networked economy, critical insights about its business exist outside the payroll.

Let’s assume the hypothetical Boeing market is open to all employees. Any individual – whether in engineering, supply chain, marketing, IT, finance, sales, or project management, and located anywhere around the globe – can weigh in on the 787 based on insights unique to their role, experience, and knowledge. All trades are anonymous, which provides the freedom to share information and honesty through the market without concern for retribution or lost face.

When would Boeing employees make trades?

Prediction markets are designed to compel trading when people believe a stock is mispriced. Since prices represent probabilities, traders act when they don’t agree with the current market prognosis. Imagine a contract called: 787 Flight Testing Will Be Ready by June 30. A supply chain employee might be the first to know that a critical parts shipment is missing and short the stock on this information. A technician might hear lunchroom chatter about a difficult instrument installation and sell her shares. An engineer may be part of a small team that has finally solved a nagging design flaw and buys up the stock on new-found confidence. A union member may short the stock, knowing that a strike is looming. A project manager may buy shares, bullish after a successful meeting with an offshore IT provider. The market will reflect all of this in real time. Would a biweekly project status report do the same? Is there another way Mr. McNerney could access or synthesize all of this disparate information and sentiment?

Why would employees trade?

Prediction markets work because traders are individually motivated to generate collective accuracy. Each time a trader has reason to believe a stock is mispriced, he or she sees a profit opportunity. The engineer privy to the design correction can quickly “buy low” before his accomplishment is widely known, confident he’ll be “selling high” before too long. The union member can sell all his shares, avoiding a major loss when the strike blows the current production schedule. These profit incentives are what make markets more promising predictors than surveys, status reports, or meetings – where timing, motives, and voices may be sub-optimal. In our Boeing example, employees wouldn’t be trading real money, but rather an internal currency that may convert to modest cash prizes, giveaways or soft benefits. Fortunately, prediction markets don’t only rely on material incentives. Organizations like Google and Best Buy have discovered the power of reputational incentives (a leader-board tracks most profitable traders) in their markets. Additionally, prediction markets are inherently democratic institutions. For many, having a voice in the system can be motivation enough.

How would Boeing executives use the market?

By tracking the various prices and price changes of these stocks, management would know exactly what employees think and feel at any given moment. For example, if the stock: 787 Flight Testing Will Be Ready by June 30 is trading at only $25 on June 10, management knows their insiders are largely doubtful the milestone will be reached. If the Will We Pass Inspection stock suddenly dives from $70 to $30, they’ll have reason to make a few phone calls. Such quantifiable confidence measures are not easy to come by, especially in real time. In this manner, the market can serve as an unparalleled early warning signal.

Intended Result: Once in possession of these unique insights, Mr. McNerney and his team could do what they do with any other form of information – take action. By adopting the prediction market as one form of decision intelligence, they could more clearly see discrepancies and investigate them … identify pitfalls and avoid them … uncover communication gaps and correct them. In a project as complex, costly, and public as the 787 Dreamliner, there are myriad ways in which better and faster information could be invaluable. How management chooses to act on the information will ultimately determine the value of the market. But the upside is significant. Improving key decisions in project management or external communications could go a long way toward restoring confidence among Boeing’s customers and investors.

As Mr. McNerney surely knows, the answers aren’t always in the room. It depends on who’s in the room! But in an organization with thousands of insightful employees worldwide, there’s a good chance the answers are somewhere in the crowd. This simple but powerful truth is why Prediction Markets could potentially change the course of events for Boeing and its Dreamliner jet. By enabling the company to swiftly and accurately tap its own collective knowledge, markets create an intelligence engine with infinite application and shelf life.

-Dawn Keller, 6.27.09