First Pitch: Finding an Accurate Trade Value For Relief Pitchers

The Pittsburgh Pirates have been connected to a lot of relief pitchers leading up to the trade deadline. Any time there is a trade rumor connecting the Pirates to a player, I usually do a Trade Value article, looking at what that player will probably cost. The exception comes with relievers. The value of relievers never matches up with the trade value that teams give up in prospects. In some cases, the price that teams pay for relievers is insane. So I could do a trade value for relievers, but it wouldn’t compare to what those relievers would actually fetch in a trade.

On Friday I looked at an example of the true price for starting pitchers in this seller’s market, using the Jeff Samardzija trade as an example. The price was $9.5 M per win, when you considered the amount in salary and the amount in prospects given up. I wanted to do something similar for relievers, to at least put some sort of number on the value that they would bring in a trade. Unfortunately, there are only two big reliever trades to go on, which means once again we have a small sample to establish the market value.

I went ahead with the values, with the disclaimer that we’re only talking about two trades, and those trades ended up very different in terms of the cost per win. More on that below.

Joakim Soria for Jake Thompson/Corey Knebel

The Tigers acquired Joakim Soria, sending right-handed pitchers Jake Thompson and Corey Knebel to the Texas Rangers in return. Soria was making $5.5 M this year, with a $7 M option in 2015. He’s pitching like one of the best relievers in the game this year, and currently has a 1.7 WAR. He’s only been above a 2.0 WAR once in his career, but has hovered around a 2.0 WAR for most of his career. I set his trade value at a 2.0 WAR for each year, which resulted in a $5.0 M value. That’s worth a Grade B hitter, or two Grade C pitchers.

John Sickels rated Thompson and Knebel as B- prospects before the 2014 season. They didn’t rank in Baseball America’s top 100, but BA ranked them #2 and #4 in the Tigers system, respectively, at the mid-season update. It’s safe to say they’d carry a Grade B rating, which is worth $7.3 M each. So while Soria was worth $5 M, he ended up landing $14.6 M.

Huston Street/Trevor Gott for Taylor Lindsey/R.J. Alvarez/Jose Rondon/Elliot Morris

The Padres traded Street to the Angels, getting what seems like the final prospects that the Angels had. This included top 100 hitting prospect Taylor Lindsey, which is a big haul for a reliever. The trade included a prospect going to the Angels, but he didn’t really have any trade value, which means most of this deal is based on Street’s worth.

Street is under control through the 2015 season, making $7 M this year, with a $7 M club option next year. I put him as a 1.0 WAR player for both years. He hasn’t been above that since 2009, when he was a 1.5 WAR player. He has an 0.6 WAR this year, so it’s safe to say he’ll continue where he’s been at the last few years. With a 1.0 WAR, and his salary, he’s worth negative $2.5 M in trade value. The Padres should have had to pay some of his salary just for a team to take him for free, based on the same $/WAR values that are used for every other player in baseball.

Instead, the Padres got a nice return. Lindsey would be worth $10.43 M as a 51-100 hitting prospect. I’d say the other three prospects would be worth a combined $4.3 M, based on John Sickels’ pre-season ratings, along with their ratings from BA. That gives a $14.73 M trade return for Street, or $17.23 M above his value.

Adjusting the Reliever Values

It’s hard to draw a big conclusion based on two trades that have different results. The one conclusion that could be drawn is that trades for relievers are way more expensive than they should be.

In Soria’s case, the Rangers got $8.5 M per win in prospect value. Unlike a lot of relievers, Soria has a lot of value, and would be worth as much as a back of the rotation starter, or possibly a middle of the rotation guy. The $8.5 M figure is still crazy, but better than the early results for the starting pitching market.

Then we get even crazier. Huston Street has a negative value, but landed $14.73 M in prospect value. The Angels basically paid $17.25 M per win, when considering the prospects given up, and Street’s salary.

This amount is probably paid because Street is a closer, and teams still pay for saves and “Proven Closers.” But that number is just insane. There’s no way that Street should be worth twice the return that Soria got. And that’s when considering that Soria — a better reliever than Street, with a better contract — also got a high return.

It’s hard to say what someone like Andrew Miller would fetch in a trade, since he’s not a closer. But when you consider how many teams are looking for relievers, it becomes hard to imagine a reasonable $/WAR being accepted by the Red Sox, or any other team selling a reliever. At the least, you could expect a return like Soria got, worth $8.5 M per win. In Miller’s case, that would be $3.6 M in value, based on his $1.9 M salary and a 1.5 WAR. That would be a Grade C pitching prospect under the age of 23, and a Grade C pitcher over the age of 23. You’re probably talking about a lower level starting prospect and an upper level relief pitching prospect. And that’s assuming the demand on Miller doesn’t drive the price up higher.

The Pirates have a system strong enough that they can definitely afford to give up Grade C pitching prospects to better the team. The question is, do they spend those pitching prospects to get a reliever for two months, or do they put those same prospects towards something that could provide a bigger impact over the long-term? The example price I mentioned for Miller is the same price that the Pirates paid for Ike Davis, getting three years of control of a platoon first baseman. Davis hasn’t been that great, but the potential upside in that kind of deal is much greater than the potential upside for a middle reliever over a two month span.

I’ve never been a fan of trading for relievers, or giving up anything of value for a relief pitcher. They’re too volatile, don’t pitch often enough to make a big impact, and the prices that are paid for them can be insane. If you look at the value Soria had, it’s very similar to the Samardzija/Hammel trade in terms of $/WAR. The difference is that I think it’s easier to find a valuable reliever for a much smaller price than it is to find starting pitchers like the two that went to Oakland. For that reason, it would make more sense to put any prospects towards a much harder position to acquire, while continuing to go the cheap route with relievers — a route that has worked so well for the Pirates in the past.

Links and Notes

**Prospect Watch: The One Thing Holding Willy Garcia Back From Being a Top Prospect

**Pirates Have Seven in MLB.com’s Mid-Season Top 100 Prospects List

**Minor League Schedule: Gerrit Cole Gets Rehab Start For Indianapolis Tonight

**Pirates Interested in Andrew Miller, Who is Drawing a Lot of Demand

**Casey Sadler blames pitch utilization on recent slide

**DSL Pirates Report: Two New Scouting Reports and an Injury Scare For a Top Prospect

**Prospect Highlights: Two Young Players From Australia Seeing Time With the GCL Pirates

Tim Williams

Author: Tim Williams

Tim is the owner and editor in chief of Pirates Prospects. He started the site in January 2009, and turned it into his full time job during the 2011 season. Prior to starting Pirates Prospects, Tim worked with AccuScore.com, providing MLB, NHL, and NFL coverage to various national media outlets, including ESPN Insider, USA Today, Yahoo Sports, and the Wall Street Journal. He also writes the annual Prospect Guide, which is sold through the site. Tim lives in Bradenton, where he provides live coverage all year of Spring Training, mini camp, instructs, the Bradenton Marauders, and the GCL Pirates.

Share This Post On
  • Don

    Tim, are AAA guys such as Mazzaro and Oliver not options at this time???

    • JayBird

      Agreed. I’m not sure why Mazzaro,at least, isn’t considered. He’s pitching well in Indy.

      • moose7195

        He pitched well last year too

      • Lukas Sutton

        His ML stat line and ability is very similar to Gomez. He is an average ML reliever. An upgrade over Frieri but not on the level of the guys like Miller

  • https://www.facebook.com/profile.php?id=5315860 ndbrian

    Tim, Did you read Dave Cameron’s piece about relievers’ value during the postseason as opposed to regular season? I don’t necessarily agree with him, but would love to hear your thoughts given the discussion above.

    http://www.fangraphs.com/blogs/joakim-soria-and-the-value-of-a-postseason-relief-ace/

    • jaygray007

      I love articles like this. Very interesting. I don’t usually think about how there is usually a day off between games, allowing you to really lean on your good relievers. I’d take Melancon, Watson, and…say… Andrew Miller against any other team’s top 3 bullpen guys.

      plus there’s the possibility that Frieri gets fixed and stops throwing meatballs (he still has a 3.48 xFIP! there is at least hope!)

  • https://profiles.google.com/101510909979106143098 David Lewis

    Dave Cameron had an article on Fangraphs a few days ago (I see that ndbrian linked the article in his comment) that provides some explanation as to why the values of relievers are so unreal from a $/WAR viewpoint. Basically, a top reliever will be getting a higher percentage of his team’s innings in the postseason, and those innings will be higher leverage. You’re not paying for one marginal win over the 70 remaining regular season games; you’re paying for an increased win probability in somewhere between 3 and 20 post-season games.

    And the marginal value of a post-season win could be immense (the difference between winning and losing the WS; the difference between going to the WS and losing the LCS…). So the value of a player that increases your likelihood of winning a dozen post-season games by even 5% could be huge.

    I still wouldn’t agree with making such trades, because I think relief pitcher performance is highly variable – sure, Koji Uehara may increase a team’s win probability in a post-season game by 5% over the long run, but a dozen post-season games isn’t “the long run”. There’s nothing to say that a Mark Melancon won’t put up a similar performance to what Uehara did, should the Pirates get that far. So teams are overpaying for comfort level, and perhaps a slight increase in probability that the new guy does better than the existing options. But even Mariano Rivera blew the 2001 World Series.

    • Andrew

      I don’t remember if this point is made in the article, but Tigers and Angles playoff odds are over 90%, and even the Angles chance of making the division series is projected at around 70%, so it is completely justifiable to pay the high cost for late inning relievers given the likelihood of the multiple post-season.

      The Pirates don’t have those type of odds, so I would say that dynamic isn’t at play, which leads back to the point everyone seems to agree on, look for marginal upgrades that don’t require much of a return.

  • leowalter

    Lots of serious,excellent opinions in these comments !

  • Thom

    Huston Street is # 2 in all of baseball in WPA, Win Probability Added.

    I’d be really interested to find out how different teams value Win Probability style data.

    • http://batman-news.com NMR

      Great question.
      .
      “The Padres should have had to pay some of his salary just for a team to take him for free, based on the same $/WAR values that are used for every other player in baseball.”
      .
      If your analysis leads to the conclusion that the Padres should’ve had to pay someone to take Huston Street, then your methodology is certainly flawed.

      • http://www.piratesprospects.com/ Tim Williams

        Just to be clear…

        A few comments up you were getting on my case about NOT using WAR to value relievers.

        Now you’re saying it’s flawed to use WAR for relievers.

        • http://batman-news.com NMR

          I’m asking for consistent analysis, Tim.

          • http://www.piratesprospects.com/ Tim Williams

            Well then, you’re welcome.

            • http://batman-news.com NMR

              Honestly, Tim, it’ll help you a lot to work on your self-confidence issues. Resorting to attempted snark instead of sticking to your arguments is very telling. People are going to disagree with you, and the job will get a lot more enjoyable once you learn to deal with that.

              • http://www.piratesprospects.com/ Tim Williams

                LOL

    • http://www.piratesprospects.com/ Tim Williams

      WPA isn’t a predictive stat. The purpose here is to show a player’s expected value going forward. Street has a very good WPA this year, but you can’t use that number to project he will have the same impact in the future.

  • piraterican21

    In a week to ten days G. Cole will be back, pushing V. Worley to the pen, don’t see a reason why he could not be the 7th inning guy. If not why not Cumpton now instead of waiting for Septmeber?

  • http://batman-news.com NMR

    Why is WAR being used as a metric for judging reliever value again? I’ve previously been told on this site that WAR is bad at evaluating relievers after I pointed out that Jeanmar Gomez and Jared Hughes are both replacement-level players.

    • http://www.piratesprospects.com/ Tim Williams

      How would you put a dollar value on relievers to determine what they’re worth in a trade?

      Also, I didn’t say WAR was bad at evaluating relievers.

      • http://batman-news.com NMR

        I would never perform such a silly excercise without knowing team financials. A dollar does not have to the value to the Pirates as it does to the Angels, or any other team for that matter.

      • moose7195

        I don’t know Tim, WAR isn’t a perfect metric, but I agree that it’s one
        of the better fitting ones for the purposes evaluating trades. I still
        don’t see how Street rates so poorly with it, even with his contract. Have you ever tried the WARs constructed by different sites, or even the WARP from Baseball Prospectus

    • Andrew

      Gomez is replacement level by any measure, but the Pirates seem to like low strikeout RHP who get ground balls, maybe the Pirates found a pitcher type that generates weak contact, not sure.

      Hughes is replacement level by FIP WAR, and has been for his career, but has been worth 1.0 wins by RA-9 WAR on the year.

      Outliers in batted ball distribution can outperform some of the assumptions of FIP, which assumes hits occur at league average rates, particularly ground ball pitchers, because ground balls don’t become extra bases at high rates. Combine that with shifting taking anyway ground balls hits and Hughes better consistency of throwing in the zone this year he has some value to Pirates, even if FIP WAR says otherwise.

      WAR isn’t prefect but the inputs are solid, and there isn’t really an alternative.

      • pilbobuggins

        Maybe your right about the pirates findind a niche type pitcher, for my money I would rather have a strikeout pitcher for the postseason. Ground ball pitchers without strikeout ability tend to get rocked in the playoffs.

  • R Edwards

    If we hadn’t given away Bryan Morris, we wouldn’t be in such a bind in the bullpen. But, since that ship has already sailed….

    I would not trade any top prospect for a bullpen rental – Miller or anyone else. One middle reliever is not going to fix all of this team’s woes. Now, if includes an extension for a guy like Miller, its worth looking at.

    Short of that, I’d get rid of Frieri and replace with Mazzaro. And I would consider the same for either Pimental or Gomez – replacing one with Oliver. Our solutions may be at Indy. It would seem that Oliver has earned a look.

  • pilbobuggins

    No arguments from me on this one tim, my stance all along has been if your going to give up prospects for pitching,go after starting pitching and I still say kennedy is the best fit for the pirates, both in need and price.

  • S Brooks

    I think WAR is too crude a metric for evaluating relievers anyway.

    First, there is wide disparity between the two systems, Fangraphs and Baseball Reference. Depending on which system you choose, J Wilson is either 1/2 run more or 2/3 run less valuable than Jeanmar Gomez, which over the small sample sizes of a full reliever season is just as likely to stay this way as it is to “even out.” (Exhibit B: by Fangraphs WAR, Ernesto Frieri and Jared Hughes are equally valuable. Let that one sink in.)

    Second, it misses on relative value among bullpen innings. WAR treats all innings the same, so it doesn’t account for high or low leverage. A mop-up guy pitching a 1-2-3 5th inning of a 7-0 blowout is not generating as much value for his team as a closer pitching a 1-2-3 9th in a 2-1 win. Or losing as much if they both give up a 2-run double. A guy pitching in the 6th who gives up the lead knows his teammates have 3-4 more chances to get it back. A closer in the same situation has much more likely lost the game. And again, relievers don’t throw enough innings for the most valuable pitchers to necessarily rack up the most WAR. WPA is a much better statistic here.

    Finally, because it’s a counting stat, it undervalues late-inning relievers relative to starting pitchers and position players, precisely because it misses the context of high-leverage. To get a true read on the value of a late-inning guy, it would be more appropriate to adjust the WAR relative to his average game leverage index because that’s how important those innings are for your team.

    I think we need to re-evaluate what a “reasonable” price is for late-inning relievers (the alternative, of course, is to ignore the deadline trade market and accept that your fatal flaw may or may not lose the game that gets you into the WC/gets you home field advantage in the WC/allows you to avoid the WC). The Pirates lack a 7th inning guy now, and it shows. No one currently on the team has been able to handle that role, and you can only hope Mazzaro, Oliver or Worley would do any better, but there’s certainly no guarantee. The average gLI for a 7th inning guy is in the 1.5 range. I would argue that if they want a guy like Miller or Bastardo, the “fair market value” would be 1.5x his nominal WAR in trade value, and another 1.4x that number based on where they are on the win expectancy curve since the transaction is being made in season (reference Nate Silver’s work on this).

    Miller’s 0.5 ROS WAR converts to 1.05 when you factor in these two multipliers, which carries a $5.25M value less his ROS salary of $0.65, or a $4.6M trade value. That would be the floor of a “stupid, but not insane” transaction. Something between 2 grade C pitchers and a grade B hitter.

    • http://batman-news.com NMR

      Very, very well done.

    • Andrew

      This is wrong.

      Baseball Reference adjust for leverage in pitcher WAR calculations.
      http://www.baseball-reference.com/about/war_explained_pitch.shtml

      Fangraphs does the same for FIP WAR.
      http://www.fangraphs.com/blogs/war-and-relievers/

      Relief pitchers, are getting partial credit for the leverage index, they do not get full credit because replacing a elite inning ace is not the same as replacing a starting pitcher or position player. There are major league quality relievers readily available to fill in. Jason Grilli’s innings are not replace by Andy Oliver/Mazzaro but Melancon. If you adjust fully for leverage index you are going to end up crediting more Wins than are available at the current replacement levels.

      To the above discussion which seems to center on the $/WAR valuations, they are not precise when discussing deadline deals not because the WAR construct is wrong. $/WAR is not as useful because the specific purpose of $/WAR is to answer the question if I had to replace a 2.0 WAR player with freely available talent what would it cost?

      Obviously, there is not freely available talent, thus selling teams can ask for greater than the free agent market rate. Also position on the win curve should change teams valuations. In addition, the article NDBrian linked to, one of the chief proponents of $/WAR, offers an explanation why the cost of elite relief aces is so high near the deadline.

      This article, accomplish the goal Tim apparently wants, he show that the cost of relievers at the deadline is high, states is is hard to accurately project cost of proposed deals, and argues the Pirates should not pay for a late inning ace and the article is much more informative than: “He’s got that closer mentality, flags fly forever, go get him” “Yep prospects fail, remember Delmon Young, and Brien Taylor.” “Make a statement.”

      • http://batman-news.com NMR

        Also very, very well done.

      • S Brooks

        Andrew thanks for linking the LI adjustment, that’s good to know and I’m embarrassed that I didn’t know that. I will maintain that there remains a lot of variance in how WAR is defined between the two methodologies, hence the nearly 1 WAR swing between Fangraphs and BBRef vis a vis Wilson and Gomez.

        I’m also not interested in revising the WAR construct. Having read the adjustments of both methodologies, the 1/2 credit for leverage and league WAA adjustment to zero work fine to keep the models internally consistent, although I’m not sold on that drastic an adjustment, as I think we’ve witnessed managerial behavior doesn’t reflect the “next man up” model of chaining so neatly (e.g., closer goes down with injury; yes, the set-up and 7th inning guys move up a notch, but rather than the 4th man in the pen taking over the 7th inning role, we often see more mixing and matching, starters going additional innings or partial innings, etc.). But as long as the WAR models themselves are still being adjusted, I will trust the process.

        I still maintain that $/WAR – which is, after all, set by buyers only and based on the actual market rates paid for actual players – has never captured relief pitching appropriately, even in the hot stove league. Just look at last year, it’s all over the map. Guys like Joe Smith, Grant Balfour and Frankie Rodriguez were getting $10M/WAR while Oliver Perez and LaTroy Hawkins got $3M/WAR. There’s no rhyme or reason to that variance – big market, small market, closer, middle innings guy – and you don’t see a 3x variance on position players or starting pitchers, where a 2-WAR guy could go for either 2/$6M or 2/$20M. Either this is a completely irrational market, or it’s only somewhat irrational but the rationale is something other than $/WAR.

        I’m not attacking Tim’s model or rationale – the exercise is noble. I just think the inputs (WAR and $/WAR) are unreliable when you’re talking about relievers. It is a hell of a lot better than betting the farm on a 2-month rental, but that doesn’t mean it’s above critique.

        Some combination of the un-regressed LI, win curve position and acknowledgment that the reliever will log a greater share of high-leverage post-season innings, I believe, are all part of what’s setting the market price.

        • Andrew

          I agree $/WAR is overly simplistic when there isn’t a free agent market, and isn’t the best to evaluate deadline moves because teams now have much more information about their projected performance than they did in the off-season.

          As far as relievers the linked article suggests that team can save money by using established closer on fixed salaries because arbitration pays relievers for saves. Thus it would be better to have Grilli pitch the 9th than Watson because it will keep Watson projected future earnings lower. Frieri got $3.8 million first time through arbitration.

          http://www.hardballtimes.com/how-paying-established-closers-saves-teams-money/

          That could explain some of the variance in the reliever market. Another part could be that projecting relievers is hard, there just isn’t a lot of data and any projection is a prisoner of the inputs.

          Overall attempting to model the trade deadline market is just absurdly difficult, you kind of have to go case by case.