The increasing popularity and use of performance analytics in the NFL has led to an alphabet soup of acronyms for advanced performance measures that can often be a bit cryptic: PFF, DVOA, EPA, QBR, WAR, CPOE- the list goes on. Sometimes these measures confirm the obvious (i.e. Patrick Mahomes is good), sometimes they reveal a well-known player may not be quite as good as his reputation would suggest (i.e. Adam Thielen in 2022), and still other times they present conflicting assessments of player or team performance (i.e. the Vikings 2022 season).

The one constant among providers of these performance measures is that they seldom delve much into how their measures are calculated, how well they correlate to actual play on the field, or the inconsistencies, limitations, uncertainty, and error that may be contained within their measures. Indeed, most advanced performance metrics are proprietary products sold on subscription, so the exact methodology and models used are closely held secrets. In some cases, an outline of the methodology is provided somewhere on a website, but that can often lead to as many questions as answers. Complicating things further, some measures like EPA can be calculated differently using different data models or weightings.

Football Isn’t Easy to Analyze

Football is the ultimate team game. Success on any given play depends on several players, maybe even all eleven, completing their assignment or at least not completely failing at it. It also depends on coaching and play-calling, and sometimes luck too. All that makes it difficult to isolate individual player performance, when frequently it is the performance of a combination of players that produces a given result- not just quarterback performance for example. And yet quarterback performance is the most scrutinized and analyzed position in football, and success of a play, a game, or a team is often focused on the quarterback. It’s perfectly valid to say that quarterback is easily the most critical position on a football team, and yet the quarterback position is also the most dependent on other players and coaches to be successful.

The same is true of other positions to varying degrees as well. A defensive secondary could struggle because of a weak pass rush or a poor scheme by the defensive coordinator. A running back may be good with the ball in his hands but suffers from poor run blocking. Or a defensive edge rusher may be good but unproductive because he faced several top tackles and/or is chipped or double-teamed consistently.

Multiple other factors can weigh on performance as well. Game plan. Weather and field conditions. Home or away. Strengths and weaknesses of a given opponent. A nagging injury.

As a result of all that, and small sample sizes of relevant data (usually the current season), most NFL performance measures offer little predictive value- the holy grail of football performance analytics.

And yet, for all these shortcomings, more advanced performance analytics can be more insightful than box score stats in describing what is working and what is not for a given team, along with a better assessment of individual player performance.

But not everyone is familiar with how each of the ever-increasing alphabet soup of NFL performance analytics is derived, and how best to use and interpret them as a basis for determining player performance. So, I thought I’d take a closer look at a few of the more widely used performance metrics starting with Pro Football Focus (PFF) grades.

PFF Grading

One of the increasingly popular performance analytics is PFF grades. I use them often in my pieces and they are increasingly found in many other pieces on player or team performance in the NFL, including in some TV broadcasts.

I like PFF grading because for most positions they grade every player on every play, which unless you re-watch a game many times, you can’t do on your own. For less high-profile positions, we often notice only the good or bad plays that stand out, rather than the complete performance of the player. And for some positions like offensive line, a player may perform well 95% of the time, but that still results in a relatively poor performance for the position.

I also like PFF grading as it’s one of the few metrics that can be applied to all positions, not just quarterback or other “skill” positions.

PFF grades are useful as descriptive measures of player performance, but they aren’t particularly predictive of future performance for a variety of reasons, some of which are common to all football analytics and statistics. And like all football stats and metrics, PFF grades have pros and cons and other things to be aware of that may or may not be shortcomings. Below is an excellent video breakdown of the pros and cons of PFF grading. It’s almost an hour long, but well worth the time to get a better understanding of PFF grading, which I’ll also dive into.

A Deep Dive into PFF Grading

How Pro Football Focus (PFF) Grades Players

PFF player grading is based on a compilation of grading the player on every play, in every game over the course of a season. A player receives a grade between -2 and +2 for every play, in half-point increments, based on their contribution on that play, with an average performance earning a 0 grade, and most falling between -0.5 and +0.5. But there are a lot of nuances that go into the grading that varies by position. The link on PFF player grading above includes links on the right side to more detailed explanations of grading for each position, but also leaves out some things I’ll address later.

The raw number grades given on the scale above for every play are then converted by PFF to a 0–100-point scale which results in the PFF grades for each player for each game. A grade of 60 is considered average. Grades in the 70s can be considered above average, the 80s good, and above 90 elite. On the other side, grades in the 50s can be considered below average, in the 40s poor, and in the 30s or lower very poor.

PFF Player Grades Aren’t Tied to the Result of the Play

It’s also important to note that PFF player grades aren’t tied to the result of the play. Instead they are tied to the player contribution to each play, regardless of the result.

For example, a quarterback who made an accurate throw to a receiver who dropped the ball may still get a positive grade for the play (and the receiver a negative one), even though the result of the play was an incompletion. Conversely, a quarterback who made a routine dump-off pass to a running back who ran it 70 yards for a touchdown may receive only an average grade for the play, as his contribution to the success of the play was minor despite it being a 70-yard TD pass on the stat sheet- and which is a big boost to a QB’s stats in other metrics.

The video below provides some detail on how PFF grades quarterbacks, to provide further context to their grading methodology:

Also, PFF grades plays called back for an unrelated penalty. For example, a running back who avoids two tackles en route to a 40-yard touchdown run, or who fumbles, may get a positive or negative grade for the play even though the play was called back for an unrelated penalty and therefore the result got wiped out.

Another example of the nuances of PFF grading: an edge rusher who went unblocked and sacked the quarterback on the play may not get as high a grade as an edge rusher who defeated a double-team block for a hit on the quarterback that didn’t result in a sack. Similarly, an edge rusher who gets a sack late in the play won’t get as positive a grade as one who does so early in the play, other things being equal. So, sometimes a defensive linemen’s pressure stats and his PFF grade may not align based on the degree of difficulty and/or level of disruption on the play.

For example, last season some Vikings’ fans were miffed because Danielle Hunter wasn’t in the mix for defensive player of the year despite having 16.5 sacks- 2.5 more than Defensive Player of the Year Myles Garrett. Garrett also had the top PFF pass rushing grade of 94.7 last season while Hunter was graded at 79.1. Garrett had 89 pressures compared to Hunter’s 80, but the real difference in grading was how disruptive Garrett was relative to Hunter by pressuring the quarterback early in the play whereas more of Hunter’s sacks and pressures came later in the play and were less disruptive.

There is a logic behind individual PFF grades not being tied to the outcome of a play, which may or may not reflect the performance of the player on the given play, but it also can lead to some disparity between PFF grading and other box score stats for the game- and some criticism of PFF grades especially when they seem low for favorite players based on box score stats.

Receivers and Defensive Backs are Only Graded on Pass Plays When They Are Targeted

Also, it’s important to note that receivers and defensive backs are only graded on pass plays when they are targeted. On non-targeted pass plays they get a zero grade. This is probably the biggest con of PFF grading- and somewhat of an exception to its mantra of grading every player on every play- as receivers who get open but are not targeted don’t get graded for doing so, nor do cornerbacks who allow untargeted receivers to get open. The opposite is also true and they fail to reward cornerbacks who have good coverage and therefore go untargeted and punish (much) receivers who don’t get open and therefore go untargeted as well. On non-targeted plays, receivers are given a zero grade regardless of whether they got open on the play or not. Each zero grade results in a slight negative to their cumulative grade.

Both defensive backs and receivers are graded on untargeted pass plays if they are otherwise involved in the play (i.e. made a tackle or block despite being untargeted) or are penalized. They are graded on run plays as well.

But, as a result of this methodology and a couple other factors, these positions are not as well graded as others by PFF. There is a rationale for why PFF doesn’t grade these positions on untargeted pass plays, and if they did their grading may have a different set of shortcomings, but nevertheless it results in grading that may not be as descriptive of actual performance as most other positions.

PFF Grades are Not Adjusted for Level of Competition

A defensive back covering Justin Jefferson is likely to have a tougher assignment than one covering KJ Osborn. Similarly, a tackle facing Danielle Hunter may have a tougher assignment than one facing DJ Wonnum. But this is not weighted or adjusted by PFF grading.

Similarly, an edge rusher or wide receiver could find themselves going up against a backup tackle or cornerback in a game that leads to a high PFF grade that can distort their performance against starting-quality competition.

So, a given player may face a higher or lower level of competition over the course of a season that can impact their grade and lead to a higher ranking that may not be fully deserved. This is also an issue with college football PFF grading, when FBS teams play non-FBS teams early in the season and players often come away with elite grades that inflate their season grades assumed to be “against SEC competition” for SEC teams.

PFF Graders Do Not Know the Player’s Assignment or the Play Call

I don’t think this is as big an issue as some make it out to be, but it certainly is the case that PFF is not privy to every play call and player assignment. Most of the time this isn’t an issue or it can be reasonably determined based on the rest of the play with a good degree of accuracy. PFF has a number of former coaches that are familiar with NFL plays and player assignments to help with grading so it’s not just some random nerd doing the grading. Sometimes now PFF gets feedback from teams later in the week on particular plays resulting in adjustments to their grading based on that feedback. But absent that feedback, there are some instances where it’s difficult to determine a player’s assignment or whether a receiver ran the right option route or whether a quarterback was anticipating the wrong route, etc. In those instances where the assignment or play call cannot be determined, PFF gives a 0 grade to those involved.

PFF Player Season Grades Aren’t an Average of Game Grades

It’s also important to note that a player’s seasonal grade is not simply an average of his individual game grades over the course of a season. Seasonal grades are compiled over the course of the season, but the compilation methodology is a bit murky. It rewards consistency, as in consistency of average or better performance, with the idea that being consistently above average over the course of a season is more difficult and desirable than more volatile game-to-game performance, and therefore earns a bonus. That compilation methodology results in seasonal grades that can vary considerably from the player’s average game grade and, for a consistently good player, can be higher than any single game grade during the season. It can also magnify the difference between two players with average game grades in the upper and lower 60s.

For example, last season Ivan Pace Jr. had an average PFF game grade of 67.4 and a season-adjusted grade of 77.1. By contrast Harrison Phillips had an average game grade of 60.1 and a season-adjusted grade of 59.7. What was an average game grade differential of 7.3 points became a season-adjusted differential of 17.4 points – a sizeable difference in grading.

You can see from these two examples of season-adjusted grades how there is a fine line, somewhere just above an average grade (60), where average game grade is increasingly given a bigger season-adjusted bonus based on above average consistency and perhaps other factors. Whether those season-adjustments are warranted and to what degree is debatable, but they can also be reflective of the difference between a good and average player often being just a couple plays per game.

In any case, it is the season-adjusted grades that usually get more notoriety, for better or worse.

Other Nuances of PFF Grading Can Be Misleading

There are other aspects of PFF grading that can be a bit misleading at times when it comes to assessing a player’s contribution to a play, but are simply difficult issues for any grading system to address. Things like a missed tackle by a defender that also forces the ball carrier into a tackle by another defender for a similar gain, or the impact and/or severity of penalties, or whether to reward defenders for tight coverage despite allowing a completion. There are some other examples as well in the video above that should be understood when it comes to the nuances of PFF grading.

Bottom Line

PFF grading is often a topic of discussion in the comment section at the DN and many other venues as well. PFF grading does have its flaws and shortcomings, as all football metrics do, and as outlined above. But they also provide a more complete picture of individual player performance than you are likely to get from any other source absent watching every play in detail of a given player yourself, and even then it can be difficult to compare or value one player to another or rank players without developing some sort of methodology of your own.

PFF grading also acts to correct misperceptions about player performance based on watching or remembering only select plays or games, rather than a whole game or season of that player, and therefore is a worthwhile measure of player performance that allows for a more complete discussion and understanding of player performance, especially given most fans don’t watch a game several times over and make detailed notes on player performance themselves. And for many positions, PFF grading is the only worthwhile metric there is that involves a more nuanced assessment of their performance, and not just box score stats, albeit one that synthesizes it down to a single number or numbers for ease of use and comparison.