The Importance of
Overall Ratings in Madden NFL Football – by HittinAgenda
Every year at this time, gamers and NFL players alike get
very excited about player ratings in the new Madden. Debates blaze across the internet about who’s
over-rated, who’s under-rated, and why everyone on “my team” should be 99 across the board. More and more, gamers are seeing the forest
through the trees and are actually questioning the real meaning of
ratings. Overall rating (OVR) certainly
doesn’t tell the whole story and savvy gamers have a sense for a player’s true
worth.
Attribute Weighting
and Positional Bias
Last year, OSer Argoos
put together a nice analysis where he determined how much each attribute
impacted a player’s OVR
([url="http://www.operationsports.com/vBulletin/madden-nfl-football/353009-what-contributes-players-overall-rating-i-have-answers.html"]link[/url]). He did this via the “create-a-player” option
in Madden 10 by increasing each attribute from 50 – 99 and recording the
resulting change in the OVR.
The same results can be determined from the known ratings in
Madden 11 ([url="http://www.operationsports.com/vBulletin/madden-nfl-football/425743-madden-nfl-11-complete-roster-ratings-vikings.html"]link[/url])
by performing a linear fit of the data.
I thought this would be an interesting analysis to see if things like
agility (AGI) were having a different impact on the OVR this year since the
Madden Development Team (MDT) has touted this to be more important with the new
Locomotion engine. My results don’t
necessarily portrait a clear picture of M10 versus M11, however, since it seems
that the create-a-player option might be a simplified version of what EA uses
to create the actual players. Here are a
few of the more interesting graphs comparing M10 “weights” to M11.
[Only admins are allowed to see this image]
There’s a ton to analyze here, so I’ll keep my observations
to a minimum to stay on topic. I’d love
to hear your thoughts though if you’d like to leave a comment below.
First, notice that agility seems to be more heavily weighted
for M11 halfbacks as compared to M10 halfbacks yet the same does not hold for
the wide receiver position. One might
think that all skill positions would be equally impacted by Locomotion;
however, it doesn’t appear that there was any conscious balancing of the AGI
weighting to acknowledge the impact of this new engine. Alternatively, the difference between these
positions might be explained by inflation to the individual AGI ratings, but
that’s beyond the scope of my analysis (mostly because I don’t have the M10
original data anymore).
Also notable in analyzing the top two graphs is that the
importance of pass blocking seems to have dropped for halfbacks while run
blocking isn’t even considered in a wide out’s OVR rating. Good blocking among skill position players
can lead to huge game changers, so a case can be made for weighting these
attributes more heavily.
The next two graphs focus on the other side of the ball
where AGI is notably omitted from the OLB position. While the MDT has discussed the changes that
player momentum has made to over-pursuing the play, it still seems that they didn’t
want this to be distinguishable via the OVR rating.
I broke the OLB position down further from Argoos’ analysis
to include ROLB versus LOLB. I’m sure
you’ve noticed that switching players from one side of the line to the other
has a concomitant increase or decrease to the OVR. This graph shows that this is likely due to
different attribute rating depending on which side the player is lining
up. As far as true ratings boost goes,
you shouldn’t expect OLBs, DEs, or O-linemen to perform differently when
switching sides other than having the potential to exploit a better matchup. I should note here that attribute weightings
at OG and OT were not different for right versus left based on the modeling
that I did.
At cornerback, I was shocked to see that my model predicts
no impact from ACC on the OVR in 2011.
This might be because there’s not enough spread in the data from the
currently available EA spreadsheets, but if not then ACC is certainly an
attribute to keep an eye on when evaluating talent at CB beyond comparing OVR. The apparent AGI boost at this position is
more in line with what I’d expect Locomotion to deliver though, based on the
lack of evidence that weighting has changed at other positions, I’m wary against crediting this difference to a
conscious change in the weighting.
Hence, it’s more likely representative of a greater spread in AGI. Unfortunately, I can’t compare the variation
without the original roster data from M10, so this remains a speculative
conclusion.
Meaning of Each
Attribute
Beyond attribute weighting, there’s also the point of what
does each attribute mean? It would be
great if there was a comprehensive instruction manual for Madden that broke
down each attribute. Right now, it seems
like we know more about [Only admins are allowed to see this image] than any of
the ratings that influence how the game is played. Speed is no longer the end-all, be-all of
ratings in Madden, so knowing how each attribute actually impacts gameplay is
critical to really understanding personnel strengths and weaknesses. Personally, I’d love to know if there was a
threshold value for certain ratings that “unlocked” animations since it would
help further customize the experience for me.
With the disclaimer that I’m not an expert by any means,
here’s my interpretation of a few of the more elusive (pun maybe intended)
ratings in the game:
Agility (AGI) – Player’s ability to make precise direction
changes thereby better capitalizing on the player’s SPD rating. I haven’t really worked out my copy of NCAA
11 yet (still playing M10 actually
), so I’m not very aware of how Locomotion
has impacted gameplay. My general
feeling is that if M11 has the nice feel that NCAA 11 has, then the community
will agree that AGI greatly impacts gameplay even if the attribute weighting is
unchanged (i.e. if there’s no impact to OVR).
Awareness (AWR) – This definition varies greatly from position
to position, but, in general, this reflects on how well the AI is tuned toward
acting like a user controlled player (and a good one at that!).
Play Action (PAC) – This attribute seems like it should have
some bearing on the types of animations that you see as well as the speed in
and out of the play action animation, but my feeling on this is that it’s more reflective how hard defenders
with lower play recognition (PRC) will bite on a play.
Blocking Attributes (PBK, RBK, PBS, PBF, RBS, RBF) – I was
surprised to see in my analysis that the generic Run Block and Pass Block still
had influence on the OVR rating with the advent of blocking strength and
footwork. I’m still uncomfortable that I
have any real understanding of what any of these attributes really do, so I’ll
choose not to speculate on how each of these impacts gameplay.
Stamina (STA) – One of the general attributes that does not impact a player’s OVR. This rating manifests both in how often a
player will have to take a “blow” and whether or not he’ll be able to finish a
long run. I’m looking forward to seeing
Locomotion in action with this attribute as the choppy, huff-n-puff animations
on long runs was a severe distraction in M10.
Toughness (TGH) – Probably the most enigmatic ratings in
Madden, this attribute is rumored to be how quickly a player will rebound from
injury. It could just as easily be how
likely a player will sustain big hits or how likely Chuck Norris is to wear Tim
Tebow pajamas.
Play Recognition – As mentioned above, I think this ties in
heavily with play action. I don’t know
whether it’s PRC or AWR that really makes the difference in a defender’s
ability to jump a route (or rather turn before the receiver does).
Press (PRS) and Release (REL) – More point, counter-point
attributes that are a little hard to really gauge with respect to their
importance. It seems like strength
(STR), AGI, and acceleration (ACC) would have more to do with winning the
battle at the line of scrimmage (LOS) for WRs and CBs, but these attributes do
contribute to the OVR rating, so they must
be important.
Are the Overall
Ratings Meaningful?
Since attribute weightings have only changed subtly, if at
all, from M10, it seems that even the MDT is acknowledging that OVR isn’t
really all that important. Yet casual
fans and Madden junkies alike crave these scores as a direct method of
comparing players irrespective of their position. The community talks in terms of the number of
90+ OVR players per team and across the league though this probably isn’t the
most significant point to debate about M11.
OVR can be a nice basis for comparison at a given position, but it
shouldn’t be the only rating examined.
There are some rather impressive undertakings by various
community members to standardize ratings (notably DCEBB2001 and his site [Only admins are allowed to see this link] irrespective of
position. This seems like a rather
obvious improvement over the current system that seems to arbitrarily increase
and decrease specific attribute ratings based on the weekly observations of one
person. Donny “The Ratings Czar” Moore
is much maligned among gamers and NFL players (including his “good friend” T.J.
Houshmandzadeh) and can easily be accused of running a popularity contest when
it comes to rating players. Personally,
I’ve always felt that Donny does a great job based on the magnitude of the
undertaking as well as the high visibility of his work. Imagine having literally hundreds of
thousands of people questioning your work.
That idea doesn’t really appeal to me, but that doesn’t mean that I
excuse the shortcomings of the system.
Allowing fan feedback to influence ratings (via the EA forums and voting
during the NFL Draft for instance) doesn’t exactly give me any warm fuzzies either.
The most appealing systems for fixing how ratings are done
include both standardization of a subset of physical attributes (e.g. speed,
acceleration, agility, and strength which can be tied to NFL Combine
performance) and calculation of performance based attributes (e.g. catching,
carrying, and throw accuracy which should be tied to on-the-field performance). Even in an improved system like this one,
there are still a ton of subjective ratings that would require film room
analysis and still be open to debate by NFL players and fans. Maybe an over-complication, but an
interesting point to consider would be a group of ratings that were calculated
based on other similar ratings (i.e. play recognition as a function of
awareness and years of experience). Still
not perfect, but at least a notable improvement.
Conclusion
OVR ratings consist of basically two important
components. First are the attribute
weightings. These weightings take into
account only a subset of the total player attributes and could be skewed for a
given position. Even worse, they may not
account for an important attribute at a given position. Second are the individual attribute ratings
that trend up and down throughout an NFL season based on the observations of
essentially one person. The combined result
is intended to give gamers a means of comparing value, but a flawed system does
little toward this end.
At a minimum, definition of attributes and what each one
actually does would pay huge dividends by providing insight for the Madden
community to provide feedback. Further
breaking down what goes into determining attribute weights and ratings might be
too much insight into the workings of the game, but could eventually provide a
better system for meaningful OVR calculations.
Breaking attribute ratings into three (maybe four if you
want to get really geeky) categories could be a huge improvement for how
ratings are done. One category could
consist solely of a player’s physical attributes such as speed, strength, and
agility. While not a perfect basis for
analysis, the NFL Combine is what the pros use, so why not use this for Madden? A second category could consist of what you
see on the field on Sunday. Performance
based calculations would drive attributes like passing accuracy up and down
without having any manual intervention required from The Ratings Czar. Finally, a
category of more subjective ratings could round out the overall ratings process
and keep an army of Donnys busy.
In the end, Madden is just a game even if we treat it with
the reverence of being a simulation of real football. Improving how ratings are determined won’t
necessarily make the game more fun and it certainly won’t decide games. The most important rating, after all, is the
user’s stick skills.
Disclaimer: Everything I know about football, I learned
from Madden.
Overall Ratings in Madden NFL Football – by HittinAgenda
Every year at this time, gamers and NFL players alike get
very excited about player ratings in the new Madden. Debates blaze across the internet about who’s
over-rated, who’s under-rated, and why everyone on “my team” should be 99 across the board. More and more, gamers are seeing the forest
through the trees and are actually questioning the real meaning of
ratings. Overall rating (OVR) certainly
doesn’t tell the whole story and savvy gamers have a sense for a player’s true
worth.
Attribute Weighting
and Positional Bias
Last year, OSer Argoos
put together a nice analysis where he determined how much each attribute
impacted a player’s OVR
([url="http://www.operationsports.com/vBulletin/madden-nfl-football/353009-what-contributes-players-overall-rating-i-have-answers.html"]link[/url]). He did this via the “create-a-player” option
in Madden 10 by increasing each attribute from 50 – 99 and recording the
resulting change in the OVR.
The same results can be determined from the known ratings in
Madden 11 ([url="http://www.operationsports.com/vBulletin/madden-nfl-football/425743-madden-nfl-11-complete-roster-ratings-vikings.html"]link[/url])
by performing a linear fit of the data.
I thought this would be an interesting analysis to see if things like
agility (AGI) were having a different impact on the OVR this year since the
Madden Development Team (MDT) has touted this to be more important with the new
Locomotion engine. My results don’t
necessarily portrait a clear picture of M10 versus M11, however, since it seems
that the create-a-player option might be a simplified version of what EA uses
to create the actual players. Here are a
few of the more interesting graphs comparing M10 “weights” to M11.
[Only admins are allowed to see this image]
[Only admins are allowed to see this image]
There’s a ton to analyze here, so I’ll keep my observations
to a minimum to stay on topic. I’d love
to hear your thoughts though if you’d like to leave a comment below.
First, notice that agility seems to be more heavily weighted
for M11 halfbacks as compared to M10 halfbacks yet the same does not hold for
the wide receiver position. One might
think that all skill positions would be equally impacted by Locomotion;
however, it doesn’t appear that there was any conscious balancing of the AGI
weighting to acknowledge the impact of this new engine. Alternatively, the difference between these
positions might be explained by inflation to the individual AGI ratings, but
that’s beyond the scope of my analysis (mostly because I don’t have the M10
original data anymore).
Also notable in analyzing the top two graphs is that the
importance of pass blocking seems to have dropped for halfbacks while run
blocking isn’t even considered in a wide out’s OVR rating. Good blocking among skill position players
can lead to huge game changers, so a case can be made for weighting these
attributes more heavily.
The next two graphs focus on the other side of the ball
where AGI is notably omitted from the OLB position. While the MDT has discussed the changes that
player momentum has made to over-pursuing the play, it still seems that they didn’t
want this to be distinguishable via the OVR rating.
I broke the OLB position down further from Argoos’ analysis
to include ROLB versus LOLB. I’m sure
you’ve noticed that switching players from one side of the line to the other
has a concomitant increase or decrease to the OVR. This graph shows that this is likely due to
different attribute rating depending on which side the player is lining
up. As far as true ratings boost goes,
you shouldn’t expect OLBs, DEs, or O-linemen to perform differently when
switching sides other than having the potential to exploit a better matchup. I should note here that attribute weightings
at OG and OT were not different for right versus left based on the modeling
that I did.
At cornerback, I was shocked to see that my model predicts
no impact from ACC on the OVR in 2011.
This might be because there’s not enough spread in the data from the
currently available EA spreadsheets, but if not then ACC is certainly an
attribute to keep an eye on when evaluating talent at CB beyond comparing OVR. The apparent AGI boost at this position is
more in line with what I’d expect Locomotion to deliver though, based on the
lack of evidence that weighting has changed at other positions, I’m wary against crediting this difference to a
conscious change in the weighting.
Hence, it’s more likely representative of a greater spread in AGI. Unfortunately, I can’t compare the variation
without the original roster data from M10, so this remains a speculative
conclusion.
Meaning of Each
Attribute
Beyond attribute weighting, there’s also the point of what
does each attribute mean? It would be
great if there was a comprehensive instruction manual for Madden that broke
down each attribute. Right now, it seems
like we know more about [Only admins are allowed to see this image] than any of
the ratings that influence how the game is played. Speed is no longer the end-all, be-all of
ratings in Madden, so knowing how each attribute actually impacts gameplay is
critical to really understanding personnel strengths and weaknesses. Personally, I’d love to know if there was a
threshold value for certain ratings that “unlocked” animations since it would
help further customize the experience for me.
With the disclaimer that I’m not an expert by any means,
here’s my interpretation of a few of the more elusive (pun maybe intended)
ratings in the game:
Agility (AGI) – Player’s ability to make precise direction
changes thereby better capitalizing on the player’s SPD rating. I haven’t really worked out my copy of NCAA
11 yet (still playing M10 actually
), so I’m not very aware of how Locomotionhas impacted gameplay. My general
feeling is that if M11 has the nice feel that NCAA 11 has, then the community
will agree that AGI greatly impacts gameplay even if the attribute weighting is
unchanged (i.e. if there’s no impact to OVR).
Awareness (AWR) – This definition varies greatly from position
to position, but, in general, this reflects on how well the AI is tuned toward
acting like a user controlled player (and a good one at that!).
Play Action (PAC) – This attribute seems like it should have
some bearing on the types of animations that you see as well as the speed in
and out of the play action animation, but my feeling on this is that it’s more reflective how hard defenders
with lower play recognition (PRC) will bite on a play.
Blocking Attributes (PBK, RBK, PBS, PBF, RBS, RBF) – I was
surprised to see in my analysis that the generic Run Block and Pass Block still
had influence on the OVR rating with the advent of blocking strength and
footwork. I’m still uncomfortable that I
have any real understanding of what any of these attributes really do, so I’ll
choose not to speculate on how each of these impacts gameplay.
Stamina (STA) – One of the general attributes that does not impact a player’s OVR. This rating manifests both in how often a
player will have to take a “blow” and whether or not he’ll be able to finish a
long run. I’m looking forward to seeing
Locomotion in action with this attribute as the choppy, huff-n-puff animations
on long runs was a severe distraction in M10.
Toughness (TGH) – Probably the most enigmatic ratings in
Madden, this attribute is rumored to be how quickly a player will rebound from
injury. It could just as easily be how
likely a player will sustain big hits or how likely Chuck Norris is to wear Tim
Tebow pajamas.
[Only admins are allowed to see this image]
Play Recognition – As mentioned above, I think this ties in
heavily with play action. I don’t know
whether it’s PRC or AWR that really makes the difference in a defender’s
ability to jump a route (or rather turn before the receiver does).
Press (PRS) and Release (REL) – More point, counter-point
attributes that are a little hard to really gauge with respect to their
importance. It seems like strength
(STR), AGI, and acceleration (ACC) would have more to do with winning the
battle at the line of scrimmage (LOS) for WRs and CBs, but these attributes do
contribute to the OVR rating, so they must
be important.
Are the Overall
Ratings Meaningful?
Since attribute weightings have only changed subtly, if at
all, from M10, it seems that even the MDT is acknowledging that OVR isn’t
really all that important. Yet casual
fans and Madden junkies alike crave these scores as a direct method of
comparing players irrespective of their position. The community talks in terms of the number of
90+ OVR players per team and across the league though this probably isn’t the
most significant point to debate about M11.
OVR can be a nice basis for comparison at a given position, but it
shouldn’t be the only rating examined.
There are some rather impressive undertakings by various
community members to standardize ratings (notably DCEBB2001 and his site [Only admins are allowed to see this link] irrespective of
position. This seems like a rather
obvious improvement over the current system that seems to arbitrarily increase
and decrease specific attribute ratings based on the weekly observations of one
person. Donny “The Ratings Czar” Moore
is much maligned among gamers and NFL players (including his “good friend” T.J.
Houshmandzadeh) and can easily be accused of running a popularity contest when
it comes to rating players. Personally,
I’ve always felt that Donny does a great job based on the magnitude of the
undertaking as well as the high visibility of his work. Imagine having literally hundreds of
thousands of people questioning your work.
That idea doesn’t really appeal to me, but that doesn’t mean that I
excuse the shortcomings of the system.
Allowing fan feedback to influence ratings (via the EA forums and voting
during the NFL Draft for instance) doesn’t exactly give me any warm fuzzies either.
The most appealing systems for fixing how ratings are done
include both standardization of a subset of physical attributes (e.g. speed,
acceleration, agility, and strength which can be tied to NFL Combine
performance) and calculation of performance based attributes (e.g. catching,
carrying, and throw accuracy which should be tied to on-the-field performance). Even in an improved system like this one,
there are still a ton of subjective ratings that would require film room
analysis and still be open to debate by NFL players and fans. Maybe an over-complication, but an
interesting point to consider would be a group of ratings that were calculated
based on other similar ratings (i.e. play recognition as a function of
awareness and years of experience). Still
not perfect, but at least a notable improvement.
Conclusion
OVR ratings consist of basically two important
components. First are the attribute
weightings. These weightings take into
account only a subset of the total player attributes and could be skewed for a
given position. Even worse, they may not
account for an important attribute at a given position. Second are the individual attribute ratings
that trend up and down throughout an NFL season based on the observations of
essentially one person. The combined result
is intended to give gamers a means of comparing value, but a flawed system does
little toward this end.
At a minimum, definition of attributes and what each one
actually does would pay huge dividends by providing insight for the Madden
community to provide feedback. Further
breaking down what goes into determining attribute weights and ratings might be
too much insight into the workings of the game, but could eventually provide a
better system for meaningful OVR calculations.
Breaking attribute ratings into three (maybe four if you
want to get really geeky) categories could be a huge improvement for how
ratings are done. One category could
consist solely of a player’s physical attributes such as speed, strength, and
agility. While not a perfect basis for
analysis, the NFL Combine is what the pros use, so why not use this for Madden? A second category could consist of what you
see on the field on Sunday. Performance
based calculations would drive attributes like passing accuracy up and down
without having any manual intervention required from The Ratings Czar. Finally, a
category of more subjective ratings could round out the overall ratings process
and keep an army of Donnys busy.
In the end, Madden is just a game even if we treat it with
the reverence of being a simulation of real football. Improving how ratings are determined won’t
necessarily make the game more fun and it certainly won’t decide games. The most important rating, after all, is the
user’s stick skills.
Disclaimer: Everything I know about football, I learned
from Madden.
