Next Gen Stats: Intro to Expected Rushing Yards

Running the football -- advancing your team closer to the end zone by using your strength, speed, elusiveness and intelligence to literally carry the ball from one spot on the field to another -- is elemental to the NFL. And yet, it's a part of the game that has been somewhat difficult for advanced statistics to capture. Until now.

Building upon the winning entry of Austrian data scientists Philipp Singer and Dmitry Gordeev in the 2020 Big Data Bowl, the Next Gen Stats Analytics team is excited to introduce a set of metrics that use player-tracking data to delve deeper into the ground game, derived from the newly developed ability to calculate Expected Rushing Yards.

The 2020 Big Data Bowl competition, powered by Amazon Web Services (AWS), offered the opportunity for data scientists, computer programmers, statisticians and inquisitive NFL fans alike to develop a model that attempts to answer this question: How many rushing yards will a ball-carrier gain from the moment of handoff? More than 2,000 data scientists participated in the open-source contest, which featured $75,000 in total cash prizes shared among the top submissions.

Singer and Gordeev, who went by the team name The Zoo, finished the competition in first place, and it wasn't close. Using the continuous rank probability score as a measure of model accuracy, the difference between Singer and Gordeev's entry and the second-place entry was the same scoring margin that separated second place and 24th place.

Let's dig into the principles behind Expected Rushing Yards and how NGS will be able to use that metric to bring new insights to a dimension of the sport that predates the forward pass:

How the model works

The team from Austria admittedly had NO exposure to American football prior to the Big Data Bowl. It didn't matter. Singer and Gordeev's understanding of the problem -- coupled with their expertise in machine learning -- allowed them to "think outside the box," as Gordeev put it in his post-competition summary of their winning solution.

Singer and Gordeev built a 2D convolutional neural network based on the relative location, speed and acceleration features of every player on the field at the moment of handoff. Singer and Gordeev's elegant solution was rooted in the fusion of their advanced understanding of deep learning and the simplification of a complex problem. Here is how Gordeev explained it on the 2020 Big Data Bowl discussion board on Kaggle, the web-based community of data scientists that hosted the competition with the NFL:

"If we focus on the rusher and remove other [offensive] team players, it looks like a simple game where one player tries to run away and 11 others try to catch him. We assume that as soon as the rushing play starts, every defender, regardless of the position, will focus on stopping the rusher ASAP, and every defender has a chance to do it. The chances of a defender to tackle the rusher (as well as estimated location of the tackle) depend on their relative location, speed and direction of movements."

While the majority of participants concentrated on feature engineering to improve the accuracy of the model's predictions, Singer and Gordeev came to the conclusion that deep learning techniques against minimally engineered features could lead to better model performance. The Zoo's convolutional neural network used only five vector features in their final model, for all 22 players on the field: X, Y, S, A and Dir, where X and Y represent location coordinates on the field, S represents speed, A represents acceleration and Dir represents a player's direction.

You can read more about The Zoo's solution here on the Kaggle competition discussion board.

The NGS team has spent this offseason implementing The Zoo's modeling architecture with the help of Amazon's SageMaker platform, paving the way for the next generation of rushing metrics. We want to thank Philipp Singer and Dmitry Gordeev for their contributions to the advancement of statistical analysis in football, and for the new discoveries that are certain to come from it.

New rushing metrics

In a complex environment like the game of football, traditional box-score statistics, like rushing yards and yards per carry, lack the context needed to reliably quantify player and team rushing performance. These stats give you only a partial picture of what happened when a rush was attempted -- the ball was moved a certain number of yards -- without telling you very much about how or why the ball-carrier succeeded or failed.

Similar to how the NGS Completion Probability model has enhanced our understanding of the passing game with derivative metrics like Completion Probability Over Expected (CPOE), Expected Rushing Yards offers a collection of derivative metrics that will enhance our understanding of the running game.

Rather than predicting a single-point estimate, the Kaggle competition asked participants to predict a distribution of rushing outcomes. In other words, what are the chances a rusher gains at least 4 yards on a play? Or 10 yards? Or 20? Generating a probability distribution is more advantageous than single-point estimates for the sake of future analysis, because a probability distribution allows us to derive probabilistic metrics, like first-down probability and touchdown probability, in addition to expected rushing yards.

Here's a breakdown of the primary metrics we've developed from the Expected Rushing Yards model:

• Expected Rushing Yards (xRY): How many rushing yards is a ball-carrier expected to gain on a given carry based on the relative location, speed and direction of blockers and defenders?
• Rushing Yards Over Expected (RYOE): The difference between actual rushing yards and expected rushing yards on an individual play or series of plays.
• Rushing Yards Over Expected per Attempt (RYOE/Att): The difference between actual rushing yards and expected rushing yards per rush attempt.
• Rush Pct Over Expected (ROE): The percentage of runs where a ball-carrier gained more yards than expected.
• First Down Probability: The likelihood a ball-carrier will gain at least enough yards for a first down from the moment of handoff.
• Touchdown Probability: The likelihood a ball-carrier will score a touchdown at the moment of handoff.

To best illustrate how each of these metrics is derived from the Expected Rushing Yards model, let's break down the 88-yard touchdown run by the Browns' Nick Chubb against the Ravens last season.

Here's the situation: The Browns led the Ravens 24-18 with about 10 minutes to go in the fourth quarter. The Ravens had just scored a touchdown and converted on a 2-point attempt to cut the deficit to 6. A false start on the Browns' opening play of the drive created a first-and-15 at their own 12-yard line with 9:47 left in the game -- and Chubb was handed the ball.

Chubb gained 88 yards on the rushing touchdown, +81 yards over expected, the second-most yards gained over expected on a run over the course of the 2019 regular season. Chubb had a 12.4% chance of gaining enough yards for a first down (15 yards), but less than a 1% chance of running for a touchdown.

How can we measure the probability a ball-carrier scores a touchdown? By deriving the likelihood the runner will gain at least the amount of yards to the end zone. Here's the probability distribution of outcomes for Nick Chubb the moment he receives the handoff:

We can derive our expected value%20is,summing%20all%20of%20those%20values.) by multiplying each of the possible outcomes by the likelihood each outcome will occur, and then summing all those values. From the moment of handoff, Chubb was expected to gain 7.3 yards on the play, represented by the line in red. The area under the curve (green) to the right of 15 yards represents the probability of gaining a first down (12.4%). The probability of Chubb scoring a touchdown on the play (88-plus yards) was less than 1%, based on the barrage of Ravens defenders in pursuit.

At the individual-play level, we can use expected rushing yards to add context to the level of difficulty of a run and extract the likelihood of specific outcomes. At the aggregate level, we can better measure the success of a player, team or type of play with new advanced metrics.

The model that changed the game

What makes Singer and Gordeev's winning model even more impactful beyond its accuracy? Because the solution uses the location and speeds of all 22 players relative to one another and does not rely on a pre-engineered feature set. Thus, Singer and Gordeev's methodology can be applied to any play, at any given timestamp.

In fact, our Expected Yards After Catch model, which we originally debuted in 2018, will be replaced with the same modeling structure as Singer and Gordeev's Expected Rushing Yards model. Just like on run plays, the new EYAC model (combined with Completion Probability) will also have the ability to estimate outcome probabilities like first downs and touchdowns more accurately than our previous iteration of the model.

This is only the beginning.