You can’t have my copy but you can buy your own by clicking on the photo.
Look, I want to get behind the “analytics” movement in football. I really do.
I read Moneyball and then saw the movie. I enjoyed both. When I was 13 years old, I purchased and devoured The 1988 Bill James Baseball Abstract. I’m wholly on board with Doyle Brunson’s logic of playing suited connectors in No Limit Texas Hold Em.
In short, all of the above to point to the value of thinking differently than your opponent. I love it.
Then what’s my problem with football analytics? Well, there are many which have been well-documented. Without getting into the “you never played the game” stuff, here are a handful of my chief complaints.
1. YOU DON’T KNOW WHAT YOU DON’T KNOW
Precise breakdowns of every player on every play of every game can be found on countless websites but “Player X failed to execute his route” and “Player Y is responsible for the B gap” ring pretty hollow when you don’t know what the players were supposed to do. Remember when Doug Flutie discredited analysts because “You watch the game. We watch the film”? He should’ve added “and we know the play calls.” Without that crucial information, your analysis is a really, really in-depth GUESS.
2. WHAT HAPPENED 45 MINUTES AGO IS PROBABLY MORE RELEVANT THAN WHAT HAPPENED 45 YEARS AGO
You want to tell me I should’ve gone for it on 4th and 2 because of the success rate of the average offense versus the average defense on the average day since the merger? Well, what if my offense is below average and my opponent’s defense is above average and the perennial Pro Bowl defensive tackle is eating my rookie interior linemen alive and my best available RB is nursing an ankle sprain? What do your numbers tell you now? Fact is– you’re not telling me what’s going to happen to me. You’re telling me what already happened to others. This is useful information and I’ll treat it as such… but that’s where it ends.
3. NOW NERDS CAN PLAY!
Football is a big, tough manly sport and analytics provide a way for all of us to get involved without getting hit. It’s appealing. We value our ability to think our way through the game and, after the fact, it makes a lot more sense to say “I wouldn’t have done what that coach did” than “I wouldn’t have thrown that interception” or “I wouldn’t have fallen for that juke move.” We can imagine ourselves in the role of coach. We all have Madden. We all have Excel. Plugging flawed data (see #1) into a spreadsheet in an attempt to accurately predict results is a fool’s errand.
4. IT’S (MOSTLY) PSEUDO-SCIENCE
I’m 100% in support of professional teams utilizing all of the data at their disposal in an effort to make the best decisions possible but it all breaks down when we get to the amateur/armchair level.
Pseudoscience is a claim, belief, or practice which is presented as scientific, but does not adhere to a valid scientific method, lacks supporting evidence or plausibility, cannot be reliably tested, or otherwise lacks scientific status. Pseudoscience is often characterized by the use of vague, contradictory, exaggerated or unprovable claims, an over-reliance on confirmation rather than rigorous attempts at refutation, a lack of openness to evaluation by other experts, and a general absence of systematic processes to rationally develop theories. A field, practice, or body of knowledge can reasonably be called pseudoscientific when it is presented as consistent with the norms of scientific research, but it demonstrably fails to meet these norms.
The lack of reliable data (primarily because of the inability to use a control group to isolate specific events within the context of an 11 on 11 game) severely injures the credibility of any conclusions derived from said information. So, the average NFL play nets 4.8 yards? How many yards is the average team attempting to gain on each play? 2? 4.8? 80? Without that knowledge, we have nothing.
5. IT’S GETTING WORSE
Looking at data before making decisions is a positive development and I hope every team I support keeps up with the times. But the further we go down this road in popular/public/amateur conversation, the more bad information creeps in. Suddenly, the word “analytics” starts to sound a lot like “run and stop the run” and “defense wins championships.” Research all you’d like but we’re getting into “He made the wrong decision and I can prove it” territory (even though you can’t) and it’s growing tiresome. The day that “Analytics Guy” realizes his information is helpful but not definitive, everybody wins.
I haven’t even hit “publish” yet and I already want to add to my list so I’m sensing a “part two”. We’ll see.
I’ve been slow to jump on the analytics train in baseball, but I definitely see it’s merits. I don’t see how it can help in football. There are too many variables to handle all at the same time. Teams would need supercomputers to crunch all the data. The biggest difference and the main reason why it doesn’t translate is that in baseball you have a pitcher that throws a ball, a hitter bats the ball, and 9 players react to that batted ball; everytime. In football it is 11 v 11 and players on both sides of the ball are running set plays and reacting at the same time to what actually happens on the field. Predictions tend to break down easily under this kind of chaos. Again, just too many variables. Then you throw in audibles, and guess what, more variables. Analytics can work for baseball teams because there are so few variables to account for in any given situation because you have at most 3-5 players reacting to the same situation. Football has players reacting to other players, the ball, the play, and referees subjective response to penalties.
So lets all be Lemmings and punt on 4th and 1 pretty much every time, If Christopher Columbus listened to everyone he would of never sailed and discovered North America, he was supposed to fall off the edge of the Earth, because according to EVERYONE the world was flat, besides that Leif Ericson had already discovered North America many, many years earlier, and I think the Indians were here before him. Ok, back to football…..we are probably more intrigued by analytics in Buffalo because of our football team, The Buffalo Bills, who are going on an absurd amount of years without making the playoffs, is it twelve years, thirteen years, who cares? I do, Where has playing safe gotten us, nowhere. Now I don’t condone never punting, but I wouldn’t shun the idea, All we are asking is to take a chance every once in a while on a 4th and 1 or 4th and 2 when we have a very youthful and healthy C.J. Spiller and E.J. Manuel and I might add are pretty strong and athletic, but that’s beside the point. I don’t care if Zeus and Medusa are on the Patriots defense, We are lowly Buffalo, we have been getting our heads smashed in by Tom Brady and the Patriots for way too long, yet when “The Moment” came to take our shot, we punted……..thanks for nothing Doug Marrone. Dick Jauron thinks your a pussy. I’d like to think we would have made a 1st down and won the game, and if we failed at least we went down swinging, instead all we have is what we have always had, that losing feeling. Worst case scenario, EJ Manuel gets stuffed or whatever, that sickening feeling stays with any true competitor until he gets another chance to redeem himself……….WE GOT NOTHING, we got a pretty good effort and as usual we got the loss, and its all because of our cowardly coach, different year same results. Whenever I’m at a casino and I’m playing Blackjack or Poker and I’m faced with a tough decision, and believe me 4th and 1 when facing an opponent that has owned you for forever is NOT a difficult decision, I like to refer to a great philosopher, from the movie Risky Business, the one and only Tom Cruise, “Sometimes you just have to say what the fuck.”
Guess what, my favorite topic BradRiter !While I agree that using statistical analysis in NFL football isn’t nearly as effective as other sports, I think you’re getting your reasons wrong (hint: sample size). I also think you’re misusing the term “analytics” and maybe creating a bit of a straw man. First, to respond to your points:
1) The fictional examples you gave were not examples of statistical analysis. “Player Y is responsible for the B gap.” is an example of the analysis of an individual play, not the application of any sort of statistical model. A more accurate example of the application of analytics might be “The coach should keep it on the ground here, since running plays behind X yard line in this quarter produce Y yards Z% of the time.” That predictive statement is based on data that is absolutely known and (for the purposes of this example) is perfectly true.
2) Statistical analysis falls apart when your sample sizes start to get small. When your sample size is one play there is no place for statistics at all. There has never been another play in the NFL with the exact same conditions of the play you’re about to make and *there never will be*. This play is a beautiful and unique snowflake. However, you still have to make a decision. Go for it on 4th and 2 or not? Unless the coach is rolling dice to decide, he’s using historical information to base his decision. Why do playcallers so often punt on 4th down? Because that’s what they’ve done in the past and (at least in their minds) that is what has worked for them. 3) The phrase “armchair quarterback” wasn’t invented to describe nerds with spreadsheets. Sport teams have always been populated by an infinitesimally small portion of the general population with outlying physical and mental traits. The overwhelming majority of us have always been locked out of actually playing the game but guys with computers were never the first to say “Coach should have gone for it on that fourth down late in the third quarter”.4) Statistics, gathered and applied appropriately, is very much real science and there is nothing pseudo about it. While it’s true individuals may be gathering bad data or applying statistical methods incorrectly, that’s the fault of the individuals and how they are misusing the math, not the underlying math itself.Regarding the lack of control groups: Creating control groups is a task performed for experimentation. Analytics is definitively not experimentation rather it’s a form of regression analysis.A properly designed regression analysis allows you to isolate variables and control for them in an existing data set. Although the variable does has to be clearly tracked in the data set to begin with. I could not regress against whether or not the players had a well-balanced breakfast that day but if I want to know if the height of the QB makes a difference in the 4th and 2 scenario above, I can easily run a regression and plot success rates against QB height metrics.5) It’s getting worse because the information is becoming more widespread and people with less of an understanding of what the data means and how to use it are becoming more vocal. This is a problem of knowledge and education though, not a problem with statistical analysis.Secondly – You’re missing the point of “Analytics”.Statistical analysis is NOT meant to prove anything. Statistics cannot be used to guarantee the outcome of any play or definitively state what any manager or coach or player should do in any given situation. Anyone who uses the word “prove” isn’t using analytics. Statistical analysis also isn’t meant to do any of the other hyperbolic things it’s often accused of like “change the game” or “give coaches a new fundamental understanding”. All analytics can do is clarify the background information that decisions makers use to make their decisions.At the end of the day, every decision made on and off the field is made a person (or group) that is using historical data to inform their choice. The ball is punted away of 4th down because of what you’ve learned in the past. The free agent wide receiver is signed because of what he’s done in the past. You’re always using past information to predict future events because you don’t know if punting will get you the win you can only guess. That receiver may turn out to be a total bust regardless of his past performance you can only guess.It turns out however, that people are actually pretty terrible at accurately collecting past information in their heads. All sorts of cognitive biases come into play. It’s what allows casinos to flourish and fleece people, even while telling them that they’re being fleeced (the odds are printed right on every table). It’s what allows Wall Street fund managers to grow rich while their clients lose money hand over fist. And it’s what allowed Billy Beane to consistently gain an upper hand on his competitors without access to any information they didn’t have themselves.The point of analytics is to provide meaningful and accurate data to decision makers. That’s all. It’s an attempt to burn through the cloudy, murky fog of our collective memory and provide hard information.
Where did all the formatting go?!
Poker has come up a couple times and I think it’s a good comparison. It’s about context. The math is very important. However, you don’t always want to push small percentages. If 51 percent of the time the result will be good but 49 percent of the time the result will be disastrous, you want to think twice and evaluate your options. Math versus feel. There are just too many variable to ever have a semblance of certainty.