Register now for - "Turbocharging Data Management and Analytics with GenAI Assistants" - Thursday, March 21, 2024 at 11:30 a.m. Eastern Time.

The Limits of Analytics: The Tale of the Red Sox, Hippos, and Groundhogs

The Limits of Analytics: The Tale of the Red Sox, Hippos, and Groundhogs

Red Sox principal owner, John Henry, rocked the professional baseball world last week when he said his team was de-emphasizing its reliance on analytics in making player personnel decisions.

A former hedge fund kingpin who made millions scrutinizing data, Henry led a group who purchased the moribund Red Sox in 2001 and then led the team three years later to its first World Series title in 86 years, thanks in large part to the team’s use of analytics or “sabermetrics” as it is known in the industry.

In his comments to the Boston Globe last week, Henry didn’t refute analytics—the Red Sox still have the biggest analytics department in baseball—but he said the team’s recent string of losing seasons made it clear that “perhaps there was too much reliance on past performance and trying to project future performance.”

The team’s coach, John Farrell, perhaps summed it up best. “Sabermetrics is always going to be a tool that is in the game to stay…. What the right balance is…. that’s where art meets science...."

The Rise of Analytics

The use of statistics, machine learning, and analytical techniques have flooded business markets in the past five years as organizations recognize that decisions based on gut feel alone aren’t sufficient to remain competitive in a fast-changing, global marketplace. The lure of using data to predict the future—from customer purchases and outbreaks of crime to machine failures and weather patterns—has fueled the rise of analytics.

But analytical cognoscenti have known for years that the best decisions are made using a combination of analytics and intuition—or art and science, if you will. An over-reliance on one or the other creates mental blinders that lead to poor outcomes.

HiPPOs 

Although gut feel or intuition can be remarkably accurate when making repetitive decisions, it is woefully inadequate as a guide for navigating new territory—which is the case for most business decisions these days.

Silicon Valley analytics guru, Ken Rudin, who has served as head of analytics at Google, Facebook, and Zynga, uses the acronym HiPPO to refer to decisions driven by intuition and gut feel. Rudin likes to say, “In the absence of data, the highest paid person’s opinion [HiPPO] rules the day.”

In other words, when business people disagree about a course of action, the boss or highest paid person in the room always wins the argument. In this context, the only way to countervail bias, arbitrate disputes, and optimize decisions is to evaluate ideas in the context of data.

The best executives use data to validate their gut feelings and those of others. When intuition and data deviate, these executives prolong the discussion. They dig deeper into the data and unearth fundamental assumptions before making significant strategic and tactical decisions.

Groundhogs

But once organizations get zealous about data, such as the Red Sox, they can go too far in the other direction. Rudin calls these organizations “groundhogs” because they are too focused on the data to see the bigger story. In other words, they can't see the forest through the trees. And a myopic emphasis on data to the exclusion of common sense leads to rather comical, if not tragic, outcomes.

Treasure Island. Rudin recounts how data led Zynga designers to initially make poor design choices in the online game “Treasure Island”. At one point, designers focused on data indicating that players abandoned the game after too many failed attempts digging for buried treasure. To fix the problem, they decided to shrink the size of the islands, which effectively reduced the number of holes players had to dig.

After several cycles of this, the designers made the islands so small that game designers realized something was amiss. So, they stopped looking at the data and evaluated their own experiences playing the game. They realized that their assumption about what motivated players was fatally flawed: players didn’t mind digging for buried treasure; they just didn’t like coming up empty handed.

To fix the problem and halt the slide in player abandonment, the designers put clues in all the holes to reward players for every effort. The clues gave players forward momentum, kept them in the game longer, and allowed designers to return the islands to their original size.

Risk Models. Being a groundhog can cause greater damage than simply losing baseball games or failing to find buried treasure. For example, the economic downturn of 2008 caught many financial services executives by surprise because they relied too heavily on flawed risk models, which didn’t have enough historical data to accurately predict the financial collapse. Intuitively, most financial executives knew that there was too much bad debt flooding the market. But whether greed or ignorance blinded them, they chose to trust the models instead of common sense. And the rest is history.

Balancing Art and Science

The best decisions don’t rely exclusively on data or intuition; rather, they balance the two.  In business as in life, blending opposites leads to better outcomes. It’s not head versus heart; it’s head AND heart. The best decisions blend analytics and experience as well as data and intuition. Decision makers must balance both. In short, they use data to validate intuition and intuition to validate the data.

Business intelligence professionals are dedicated to providing business people with data to make accurate and timely decisions. But we should remember that data is not the only ingredient in good decisions. Besides showing the numbers, we need to provide business people with collaborative mechanisms that make it easy for them to share and discuss  data with others.



Wayne Eckerson

Wayne Eckerson is an internationally recognized thought leader in the business intelligence and analytics field. He is a sought-after consultant and noted speaker who thinks critically, writes clearly and presents...

More About Wayne Eckerson