If you’ve read this site for any amount of time, you’ve probably noticed that we use the WAR stat. A lot. The trade values series is based off WAR (although that wasn’t originally designed by us). We point out WAR when talking about the overall value of a player in a given season, or to compare two different players. We use WAR when estimating what a player will receive through the arbitration process.
The WAR stat keeps baseball fans split. Some like it, and some hate it. You can usually tell who is in the second group. They’re the ones saying “WAR, what is it good for?”, like that’s something that hasn’t been said on every message board ever, three times a week since WAR became a popular stat. Earlier this week, Dave Cameron wrote a story on WAR at FanGraphs (titled What WAR Is Good For). The story was in response to an article on ESPN.com criticizing the WAR stat. And now I’m sharing Cameron’s article because he says a lot of great things about the usage of WAR.
First, Cameron leads off by pointing out that WAR is too often used to end discussions, rather than promote them. He says that “WAR was never designed to be the only statistic that matters, nor should we view it as some kind of infallible truth.” He then goes on to point out that all stats provide an answer to questions, and that WAR is aiming to answer the question that is asking “how good is that player?”
Cameron’s whole article is a good discussion on WAR, the value of the stat, and what it is trying to show. I especially like the part where he goes into detail what the Wins stat of a pitcher is really trying to answer. I’m not going to continue discussing the merits of WAR, because Cameron covered it. What I will do is use this as a defense for advanced stats, and the resistance to accepting their value.
Just like WAR, there are other advanced stats we often use on this site, and a lot of them get similar reactions to WAR (usually from the same crowd that is against WAR). I don’t even look at fielding percentage for a fielder. Instead I rely on UZR. I put a pitcher’s ERA out there for reference, but I’m always sure to include his xFIP (Expected Fielding Independent Pitching), and/or his K/9, BB/9, and HR/9 ratios. I’ll also dive a little deeper to look at his Batting Average Per Balls in Play (BABIP), his home run per fly ball ratio (HR/FB), and his strand rate (LOB%).
Earlier in the off-season there was a debate about the value of Mark Melancon. On one side you had people looking at Melancon’s horrible ERA in 2012. On the other side, you had people looking at his 22.2% HR/FB ratio, his 59.4 LOB%, and his 3.45 xFIP. The people looking at his ERA didn’t like the move. The people looking at his advanced numbers saw a lot of bounce back potential. The league average HR/FB ratio is around 10%. The league average LOB% is around 70%. That means Melancon was giving up an unlucky amount of home runs, and was unlucky in stranding runners. It’s almost the exact same situation as Joel Hanrahan when he came to the Pirates in a trade.
If Melancon does bounce back this year, the people looking at the advanced numbers probably won’t be that surprised. As one of those people, I’d be more surprised if he repeats his advanced metrics from 2012 and gives up a 22.2% HR/FB ratio again (his 2010 and 2011 ratios were 9.1% and 11.1% respectively).
Melancon is one example of a split between old stats and newer advanced stats. Clint Barmes would be another.
Barmes hasn’t been the most popular player in Pittsburgh. He was signed to a two-year, $10.5 M deal. He came in and put up horrible offense in 2012. Statistically, his defense was great. According to his UZR/150, Barmes ranked second in the majors among 21 qualified shortstops. He was one spot behind Brendan Ryan, who is another all-glove shortstop.
The main part of his value is his glove. But that value comes under another debate over the advanced stats. For those that don’t trust UZR, and rely on the eye test, Barmes isn’t anything special. To be fair, it’s not just UZR that considers Barmes a strong defender. He ranks near the top of every advanced defensive metric. I find UZR to be the best of those, which is why I always use that stat.
The underlying complaint that seems to come up with every advanced metric is that they’re not perfect. That’s true, but if we discounted all stats because they weren’t individually perfect tools for overall analysis, we wouldn’t have any stats left. Too often the error is “This new stat isn’t perfect, so let’s keep using this old stat that has been in place for years”. Determining whether the advanced stats are perfect draws the attention away from the real goal of advanced stats. They’re not meant to be perfect. They’re just meant to be better.
For years people have used ERA and W/L records to determine the value of pitchers. Then others came along and determined that W/L is more about what the team did than what the pitcher did, and that there were way too many factors that went into ERA, such as the fielders around the pitcher, luck, and park factors. So they discovered things like “how often do pitchers strand a base runner” or “what is the percentage of fly balls that leave the park”. They also managed to figure out a way to remove the impact of fielders and get FIP, or xFIP if you also normalize the HR/FB ratio. In the end you go from a basic stat that just tells how many runs a pitcher gives up per nine innings — disregarding any other factors — to a stat that digs deep into how good that individual pitcher is by removing those other factors.
Then there’s UZR and the eye test. “Eye test” fans say that they know a good fielder when they see him. They watch the games, they see the plays that are made and the plays that are missed. They’ve watched enough baseball to know what is an easy play and what is a hard play to make. They can use that knowledge to determine whether the player in front of them made a difficult play, or missed a hard play. When you break down what the eye test is, you see the flaw.
What are you doing with the eye test? To start with, you’re watching a play. You’re making a mental note of the outcome of that play, and everything that happened to produce that outcome. You add that single play to every other play that you’ve seen, and you then compare it in your head to your memory of every other play that you’ve ever seen. After comparing that play to every other play you’ve seen for roughly a second, you determine how good the play was. You do this for every play that you see that individual player make, and after a certain amount of plays you have enough data to say whether the player is a good or a bad fielder.
UZR does the exact same thing, only in a way that humans can’t. The process for UZR is to divide up the field into zones. Every time a play is made in any game, the zone that the play was made in is recorded. If Clint Barmes makes a play between third base and shortstop, he gets credit for making a play in that zone. The way his UZR is calculated is by taking the percentage of plays he made in each individual zone, and comparing that to the percentage of plays that every shortstop made in those individual zones. If Barmes makes 10/10 plays in one zone, and the average shortstop makes 8/10 plays in that same zone, then Barmes gets a favorable score for that zone (that’s not the formula for UZR, just a number used as an example).
UZR does what the eye test is supposed to do, but can’t. In the second that it takes you to determine whether a play is good or bad, it’s impossible for you to remember every play you’ve ever seen, take the percentage of plays made in that specific zone, and then figure out how the percentage of plays that the player you’re watching has made in the same zone. You’re just reacting to how flashy the play looked. You don’t focus on whether it’s been Nate McLouth’d — where a simple play turns into a highlight reel play. You don’t consider that a guy with good range just missed a fast grounder, while a guy with poor range wouldn’t have gotten close to the ball. That’s what UZR does. It records every play from every game, and compares all of the stats to determine how good each individual player is. A person would have to watch every play of 162 games a year for 30 years to get that kind of data. Then there’s the issue of computing it all from memory.
I agree with Cameron that advanced stats aren’t the end of the discussion about players. I do think they provide the smartest way, for now, to have that discussion. That’s not saying that people who use advanced stats are smarter. It’s just that the arguments those people are making are supported by smarter stats that were designed to be better than the traditional stats. The stats aren’t perfect, but they are the best we have now. In a few years, Field F/X will probably provide us with a huge upgrade over UZR for fielders. And that just goes along with the goal of advanced stats: always try to improve on the previous methods of evaluating players. It’s impossible to do that if you never move beyond batting average, RBIs, won-loss records, or fielding percentage.
Links and Notes
**The 2013 Prospect Guide is now available. The 2013 Annual is also available for pre-sales. Go to the products page of the site and order your 2013 books today!
**Keith Law Releases His Top 10 Pirates Prospects.
**Here is Dave Cameron’s article again: What WAR Is Good For
