Communication, Leadership and Analytics

In the recent series on the future of sports analytics that I coauthored with University of San Francisco Professor Vijay Mehrotra (Part 1, Part 2, and Part 3) we wrote about the importance of communication and leadership. The importance of communication and leadership in the success of an analytics program was made abundantly clear through the results the Sports Analytics Use Survey that I conducted over the last several months. 27 individuals representing teams from the National Football League (NFL), Major League Baseball (MLB), National Basketball Association (NBA), and the English Premiership League (EPL), answered questions on their teams use of sports analytics. The survey provided significant insight into how teams are utilizing analytics and some of the problems that they are running into. It also provided an interesting example when two executives from the same team answered the survey. One of the executives was in the personnel department and the other was in the information technology (IT) department. This is a team that has clearly made some investment in analytics, and the personnel executive was clearly interested in how sports analytics could help his team gain a competitive advantage. 

An examination of the responses from these two individuals demonstrated that, even teams that are interested in developing an analytics program can end up not fully leveraging their investment if the lines of communication between analysts and decision makers are not wide open. These two executives, working for the same, relatively small organization had radically different views of the state of their team’s analytics program. The table below contains some of their responses and the conflicts are obvious. The two executives had very different ideas about how data is used and accessed within the organization.

 Either the IT executive was wildly optimistic about the state of the team’s use of analytics, and/or the personnel executive was simply unaware of the capabilities of the team. In either case though, what is clear is that integration of the analytics program into decision making was not happening. The team had not leveraged their analytic investment into a competitive advantage in part because, as these responses demonstrate, these two areas of the organization did not communicate. Lack of communication around basic concepts such as whether quantitative information has had a significant impact on the decision making process, indicates that the organization did not have a clear plan for how to utilize the tool of analytics. This was made very clear by their responses to the statement: “Your analytical capabilities are stronger than your competitor's.” The personnel executive answered “Somewhat disagree” while the IT executive answered “Strongly agree”. This extreme difference in opinion is a symptom of missed opportunities to gain a competitive advantage.

No matter how insightful or complex analysis is, it is totally wasted if it is not communicated effectively and in line with a clear analytics plan established by leadership.

Beyond 'Moneyball': Part 3

Professor Vijay Mehrotra and I wrap up our three part series in Analytics Magazine with this:

http://bit.ly/AENiBD

Beyond 'Moneyball':

The first a a three part series I am writing was published in Analytics Magazine.

Beyond ‘Moneyball’:
The rapidly evolving world of sports analytics, Part I

 By Benjamin Alamar and Vijay Mehrotra

Over the past few years, the world of sports has experienced an explosion in the use of analytics. In this three-part series, we reflect on the current state of sports analytics and consider what the future of sports analytics may look like.
We define sports analytics as “the management of structured historical data, the application of predictive analytic models that utilize that data, and the use of information systems to inform decision makers and enable them to help their organizations in gaining a competitive advantage on the field of play.”

(http://www.analytics-magazine.org/special-articles/391-beyond-moneyball-the-rapidly-evolving-world-of-sports-analytics-part-i.html)

Initial Poll Results

As data starts to arrive from the entirely unscientific poll posted below, one result jumps out immediately. The very first question of the poll asks which area within analytics will have the biggest impact on teams. The choices offered were: Collecting new data, better data management, new metrics, integration of statistical analysis in decision making, and better information systems for decision makers. In writing that question I had imagined that the "new metrics" answer would be fairly popular. That is the area that draws most practitioners initially, as it is the use of statistical tools to measure and learn about a sport. The results did not at all match that expectation.

The chart above indicates that integration of analysis and better information systems are the areas that the respondents felt would have the biggest impact, while none of them selected new metrics. What this result suggests (and again, this was hardly a rigorous poll) is that the area of analytics that is perceived to be the biggest area of growth is finding ways to help decision makers use the analysis that has already been done. This can take the form of better reporting, more in depth conversation, better explanations, and/or better information systems (that put the decision makers in control of the analysis to some degree), to name a few.

Still time to take the poll if you have not already.

Additcted to Math?

My first reaction after reading Jonah Lehrer's piece on Grantland was to try and ignore it. It's the same debate we've been having since Lewis published Moneyball right? But there is something different in this latest railing against sports analytics (and please can we stop referring to statistics applied to any sport as Sabrmetrics? That is a baseball term, nobody in basketball or any other sport refers to their work that way.) The difference here is that instead of being accused of not being relevant, us geeks are being accused of being too relevant. Much as my grandfather accused me of ruining sports, Lehrer is accusing decision makers in sports of ruing the game because they only care about the numbers.

This is a rather bizarre charge, as any of who have worked for teams not named the Houston Rockets can tell you. Decision makers (GM's, coaches, etc) are not exactly waiting breathlessly for the latest pronouncements  from their resident geeks. Do some teams factor good analysis into their decision making? Sure they do. By my count 11 of the 16 NBA playoff teams employed analysts, at least to some degree, but all of those teams also have serious scouting departments that employ a lot more people, and the scouts are not ignored. Personnel and coaching decisions are looked at from every angle imaginable, and it is rather silly to suggest that somehow math has put all of these other sources of information in the back seat.

I do want to address two specifics of the article though. The first is the use of the Mavericks as the counter example and the second is the phrase "The underlying assumption is that a team is just the sum of its players, and that the real world works a lot like a fantasy league." and both points actually tied back to the same idea.

First the Mavericks: The Mavericks are one of the most innovative teams when it comes to analytics, employing the first statistician who actually travels with the coaching staff and works with them on a daily basis. Roland Beech is one of the best statisticians in the world when it comes to basketball, and while I do not know what Roland's numbers looked like for Barea, I can bet that he had  input on the decision. Data may not have made the decision, but, as it should be, was a factor. To continue on the Mavericks example, Lehrer notes that "According to one statistical analysis, the Los Angeles Lakers had four of the top five players in the series. The Miami Heat had three of the top four." (cleverly linking back to Beech's own site 82games.com to make his point). The problem is that not all analysis is created equal. What is published on the internet may not always be the cutting edge of basketball analysis (or football or baseball or soccer ....). Wayne Winston once famously said that, because of his analysis, we would advise a GM not to sign a young Kevin Durant at any price. Just because some one puts a number down does not mean it is a valuable number.

On the second point regarding the assumption that a team is somehow the sum of its parts, I would suggest that most GMs, Coaches, and even statisticians are sophisticated enough to know that  this is simply not the case. I have never heard a serious statistician argue (outside of baseball at least) that you can simply add metrics together and get a result that predicts the outcome of adding a specific player.

What this comes down to, I believe, is a general misunderstanding about how sports analytics is both practiced and utilized. Sports analytics is, at times, a set of sophisticated tools that can help provide insight into the games we love. It can even be applied, as Dean Oliver has, to issues like team fit, but any good statistician will also be the first to explain to a decision maker the limits on the analysis. Are their people that take their analysis too far and draw conclusions that are not supported by their own work? Of course, their are irresponsible people in every profession. That does not mean, however that the tool is being over used or is in any way shifting a decision maker's focus away from the important variables.

Let us not have this debate any longer. Live and let live. Statisticians, scouts, fans, coaches and  general managers can choose how much stock they put into various types of analysis, but lets not dismiss an entire field that is, honestly, still in its infancy.

Impact of defensive pressure, distance on pass completion

The data from 2010 Brazil Serie A provides data on passing that has not been previously available. The first rule of good analysis is that when you get new data, the first thing to do is to look at the data. So I started by generating a historgram of distance of all 118,445 passes in the data set. The result (below) is much as expected with the mean distance of pass being 21.6 yards, with more passes below 20 yards than above 30 yards.

Click here for the rest of the article.

Analytics and Communication

In his excellent book on the research around the theory of Deliberate Practice, Geoff Colvin distills volumes of often complicated academic research into a clear understandable prose that is easily accessible to a wide audience. Colvin uses a series of carefully chosen anecdotes to explain the various dimensions of this complicated theory (yes, there is more to it than 10,000 hours by the way). What is remarkable about the book, is that it accurately reflects the messages and strength of the research that has been done. There  are varying levels of evidence from the research for different aspects of the theory, and those points are clearly made. The goal of this post is not to sell more copies of Talent is Overrated (though I have provided a link in case you are interested), but rather to make the point that clearly communicating research is at least as important as the work itself. This is just as true for statistical analysis as it is for scholarly research.

At the 2010 Sloan Sports Analytics Conference, I was sitting next to a high level NBA executive at a research presentation. The work being presented was interesting if not revolutionary. When the presentation was over, I went to the front of the room to ask a few questions of the presenter, but was beaten to the punch by the exec I was sitting with who said  (and I paraphrase here a bit) "oh my God, you can talk". And it was true, the presenter had distilled some very complicated analysis down to the core message and accurately conveyed the strengths, potential, and limits of the work in such a way that audience could clearly understand it. If the presenter had not been able to do communicate his research to a non-geek audience (ok, it was SSAC 2010 non-geek is overstating) he would not have been speaking to a full room by the end of his presentation, never mind having extended conversations with NBA execs about it.

Communicating statistical analysis is a careful balance between the strength and limits of the analysis. The story of the analysis has to be conveyed in such a way that a non-geek user of the information can use the analysis properly. Your projection may show that a player is going to improve their rebounding by 20% over the next 3 years, but you also need to convey the risk associated with that analysis. What are the range of likely outcomes? What are the risks?

It is tempting when when working with team executives to make it all too simple and speak in absolutes, especially when others are making similar statements about their point of view. It is incumbent on the analyst not to fall into this trap though, because our analysis does contain variance and we will be wrong. When we are wrong, it becomes easy to dismiss the analysis if we have spoken in absolutes, while if we have strongly communicated (and accepted ourselves) what are research actually says, then, while we may not win every argument, we will win more over the long term.

It is also possible to believe that we are so clever in the techniques we have used to solve a problem, we lose sight of the problem we were trying to solve. It is rare that you will run into an exec who understands or truly cares about how cutting edge the techniques are. They want to know that they are getting good information that they can have confidence in, not that you used some slick new neural network algorithm in R to get the slickest results. This is one of the reasons the communication piece can be so tricky. We have confidence in our results because of the techniques used, and while you may want to have the one sentence description of what you actually did ready in case they ask, management will only have confidence by seeing the results.

After spending hours and hours carefully constructing your analysis, be sure to put a significant amount of time into deciding how to present it. Think like your audience, and what will help them use the analysis properly. If you don't communicate the analysis effectively, then the analysis will be wasted.

Why Sports Analytics?

The "Why" of sports analytics is actually fairly obvious: good sports analytics can help a team win more frequently and more consistently. The real question with sports analytics is not "why?", but "how?" By how I do not mean how do we set up a good sports analytics program - this is a vital question, but not what I am tackling today. Before you determine how to implement a sports analytics program, you first have to understand how sports analytics can help your team win.

Sports analytics can help a team win more frequently and consistently through three general functions: New Information, Synthesis of Information, and Efficient Information. Each team will emphasize a different function of sports analytics based on their core competencies, but a comprehensive sports analytics program will involve all three.

Functions of Sports Analytics

 New Information
The classic use of sports analytics is to provide new information - usually through statistical analysis of performance or game data. This function uses raw, quantitative data to create new information. Team's in all sports have done this privately, creating metrics and projections that help them better measure current and future performance of their players as well as how to best utilize the players they currently have. The power of new information is obvious when compared to the investment community. Investors that have information that other investors do not about the future earning power of a company have a competitive advantage, this why there are laws governing insider trading (hello Martha Stewart).

New Information & the Draft
In sports, if the draft is akin to acquiring an asset in the investment market, than knowing more than about the future potential of the draft eligible players puts your team at a competitive advantage, and more likely to out perform your draft slot - that is you are more likely to draft a player who will perform above the average player selected at that position. One example of this is Chase Budinger and the Houston Rockets. The Rockets are know to have the most advanced analytics program in the NBA and in the 2009 draft they put that program to use. The Rockets traded a future 2nd round pick to the Detroit Pistons for the 44th pick in the draft and selected Chase Budinger. In less than two seasons, Chase has logged over 2700 minutes, hits 35% of his 3s, and has become a regular starter. For an idea on how significantly Chase has out performed the 44th pick, previous players drafted 44th overall include: Reyshawn Terry (yes, that Reyshawn Terry), Tim Pickett, and Lonny Baxter. Houston used their analytic draft system based on college data to provide them with better information on Chase's probable impact on the NBA.

Synthesis of Information
Sports teams, like all organizations, are filled with a many different types of information. There are vasts amounts of quantitative data such as player and team performance metrics, qualitative information such as scouting reports, and hyrbid information such as medical reports. Typically these all reside in various hard drives, spreadsheets, disjoint databases, and word documents. Synergies between the various types of data often exist. That is, putting financial information together with performance data does more than just bring to two types of information together, it creates a new type of information on the value of the player. Good analytic systems bring these various types of information together not just to house them in one place, but to combine them and extract all of the synergistic value out of them.


Efficient Information
Top decision makers with pro sports teams may actually be the busiest people on earth. They are constantly being bombarded with information and requests for their time. Agents, players, fans, owners, and other related parties are always looking to get some time with the top guys and each game, workout, practice, news event etc create more information for them to consider. As Dean Oliver said "stats can see all of the games" which means that decision makers, if they have the time, can consider all of the games. These executives have deep knowledge about their sport, and no one is in a better position to leverage the information that exists into a competitive edge than they are, the limiting factor on them is generally not resources or information, but time. Good information systems give them back some of their time, by creating a more efficient process for them to interact with the information. If an efficient analytic system can easily save a decision maker 10 to 20 minutes a day, that translates into 5 to 10 extra hours a month. That is extra time to spend watching more film, talking through more trade options, exploring additional strategies, and processing even more information. These decision makers will gain a competitive advantage by being able to accomplish more than their adversaries in the same amount of time.

New, better, and efficient information are paths by which a careful investment in sports can yield more wins. It has been estimated for non-sports teams, that an investment in an analytic system with a statistical component is approximately 145% (Davenport & Harris, 2006). While there is no comparable estimate for sports teams, it is not hard to see how analytic systems can help create more wins and keep winning teams on the winning path.