The fog of glove: how to measure defense?

the full SABR Defensive Index™ rankings, through games of July 12, 2015.

http://sabr.org/sdi/2015-07-12

Do you guys think putouts per inning or putouts per nine innings is a decent way of assessing an outfielder's defensive value over an entire season?

out of date

Obviously errors and assists are not a part of the stat, but other than that, I don't know what is missing.

I'll throw out the example that's bothering me: Matt Kemp 0.6 WAR for the year, Gregory Polanco 2.5 WAR.

Kemp .265/.312/.443 23 HR, 100 RBI
Polanco .256/.320/.381 9 HR, 52 RBI

In right field this year:

Kemp 1282 innings, 269 putouts (8 errors, 10 assists)
Polanco 1220 innings, 247 putouts (8 errors, 13 assists)

I'm assuming Matt Kemp's lower WAR is due to defense, but he has a higher number of putouts per inning than Polanco. What is being measured that accounts for the difference in WAR?

not really sure. maybe Kemp had a lot more balls hit towards him that he didn't get to?

there's also the base running component of WAR tho, that would probably better explain the gap.

Looking at the fangraphs stats, Polanco does have more OOZ putouts - 97 vs. 71 for Kemp. They have almost exactly the same percentage of putouts per ball-in-zone.

26 more OOZ putouts gives a guy with much worse hitting stats four times the WAR as the other guy?

Polanco does have 27 stolen bases and Kemp has 12. Kemp has 38 more total bases, though.

Polanco has a 5.3 baserunning, while Kemp is at 0.9

Kemp 1282 innings, 269 putouts (8 errors, 10 assists)
Polanco 1220 innings, 247 putouts (8 errors, 13 assists)

these are not good stats to evaluate defensive value. especially errors. scorekeepers don't generally give out errors for plays where a really slow outfielder (or one who takes bad routes or gets bad jumps) comes nowhere close to catching the ball. if they did, kemp would have like 160 errors! (sorry matt kemp)

you're using baseball-reference WAR, i think? i'm much more familiar with fangraphs, but i'd just dig into the glossary/FAQ for bWAR and figure out how they calculate defensive and baserunning value.

Polanco has a 5.3 baserunning, while Kemp is at 0.9

Kemp was the better offensive player by far. Thirty-eight more total bases, I think, means more than these numbers. So does forty-eight more RBIs.

scorekeepers don't generally give out errors for plays where a really slow outfielder (or one who takes bad routes or gets bad jumps) comes nowhere close to catching the ball.

But I mentioned just above that Kemp has pretty much the exact same percentage of putouts per ball-in-zone that Polanco does according to fangraphs.

By the way, fangraphs WAR for these two is Kemp 0.4, Polanco 2.3.

there does seem to be more of a disparity between the two than meets the eye initially. but i think you were on the right track before: going by FG's revised zone rating, the two converted outs equally as often for balls in their zone (.888 for kemp and .882 for polanco). that doesn't tell us much though, since balls that are in the zone, especially for outfielders, generally aren't difficult plays and there is not going to be a huge difference between the best and worst players. (kevin kiermaeir, who by some metrics had the best defensive season of anyone in about 10 years, had a revised zone rating of .928, just 3.5 points higher.) so there isn't much to be gained by being really good at catching routine fly balls, since just about anyone can do this. where outfielders can add value is by getting balls out of their zone -- balls that normally shouldn't be caught and would tend to go for doubles and triples. despite seeing significantly less action -- polanco had about 40 fewer balls hit into his zone than did kemp despite comparable innings -- polanco managed to turn 97 balls hit out of his zone into outs, compared to 71 for kemp. so again, on the balls that actually matter -- the ones that would otherwise go for extra bases -- polanco made 26 more outs.

this is obviously a very simplified answer, but it's the basis for advanced fielding stats. other things, like strength of the pitching staff, home field, etc are factored in too

Do some balls out of the zone fall in front of the outfielders for singles? I agree 26 hits is a lot, but if half of them are singles...

Ultimately, I think you've got quite a long ways to go to pull back 60 points in slugging percentage, 14 more home runs, and 48 more RBIs. I'm not sure that 26 more hits allowed does that. Or even comes close? Of course, it not only has to pull it back enough that they're even, but enough that Polanco ends up with a significantly higher WAR.

you gotta stop saying the r-word

kemp was not that much better at the plate. he hit for a .750 OPS to polanco's .701, good for a wRC+ of 109 to polanco's 94. meaning kemp provided 11% more value at the plate than the average hitter, and polanco 6% less. taking into account base running (which favors polanco), kemp was worth 8.1 runs above average offensively to polanco's 1.0. maybe about half a win. polanco was worth about 25 more runs in the field, by FG's calculations. sum those, and you have about a 2-win difference

sorry, that should say kemp provided 9% more value than the avg hitter

But I cannot see where 25 runs comes from. I can see 26 more hits allowed. Actually, I'm not sure that Kemp allowed 26 more hits. Kemp played about seven more games than Polanco and actually had 22 more putouts total. Maybe those are outs most players make but Kemp was there, recorded the putouts, and that has some value.

Most hits are hits to the outfield and not infield hits. So maybe those 26 hits are average hits. Even if they're not, they don't equate to 25 runs. If they're average hits, they maybe equate to something like ten runs.

As for RBI's, Polanco is a leadoff hitter and it's not a fair comparison. He had 83 runs scored and Kemp had 80. Polanco, with his 52 RBIs, did have good numbers with RISP this year. As did Kemp, as hitters with 100 RBI, you would think, generally do!

yeah, i can't exactly explain where the 25 runs came from, that would be a question for dave cameron or somebody. there are more layers and adjustments that go into it than the stuff i mentioned, though that's the main idea/backbone of it. but to answer your original question, errors and putouts are not a good measure to evaluate fielders, because the extent to which they take into account range is limited. as far as these two players go, polanco was marginally worse at the plate (though some of that was mitigated by superior base running), but far better in the field. it's certainly possible that the metrics overvalued his edge on defense, but the fact that both systems came to the same conclusion makes it more likely than not that polanco was a more valuable player overall this year

I don't understand wRC, but if you use James' runs created stat, Kemp had eight and a half more runs created than Polanco (80.3 vs. 71.8). So, that's about .85 of a win if we're using ten runs for a win. To account for a 1.9 difference in WAR, then, Polanco has to to make up 2.75 WAR in difference.

By having 26 more putouts.

Polanco has to make up 2.75 WAR difference in defense, I meant

And I'm sorry if I'm being stubborn, I understand that "more layers and adjustments" involves ways of trying to measure all of those outs - maybe how hard they were hit, how much time the fielders had to get to the balls, etc. But, again, there is such a lot of ground to be covered to account for 2.75 WAR given Kemp's better offensive numbers, same defensive numbers on balls in the zone, and maybe not that big of a difference on balls out of the zone.

Or, sorry, to account for 2.75 WAR just given the defensive differences

well the 26 outs are kind of a lot, given that polanco saw much less action in general. the other adjustments could include things like park factors, whether their respective pitching staffs give up a lot of ground balls vs fly balls, idk, other stuff. but if you're going to make an effort to understand the game, you have to be willing to admit that defense is more important -- and more complex -- than previous generations have understood. it's not difficult to envision a situation where two averagish hitters -- which polanco and kemp were this season -- could be separated due to differences in defense and base running. it's really no different than, say, mike trout being worth a couple of more wins than miguel cabrera, even though the two are both great hitters

but again as to why in this specific case it turned out this way, i couldn't tell you the exact reasons. but i do think that skepticism that a case like this could actually happen is grounded in an old-fashioned belief that defense is of negligible importance

http://www.fangraphs.com/library/defense/uzr/

here is a decent primer on UZR, the defensive metric FG uses, as well as why we should be using these in the first place

I certainly do not believe that. I'm just stating that the difference that I am seeing defensively is 26 outs. Polanco didn't see much less action - 1282 innings vs. 1220. And I'm not sure that matters anyway because Kemp played more, recorded outs, and was therefore valuable in those 62 innings that Polanco didn't play. But again, 26 extra hits for the other team, if they're average hits, should be worth about 10 runs or so. Given Kemp's 8.5 advantage in runs created, Polanco should then beat him by something like 0.15 WAR and not 1.9.

it's not difficult to envision a situation where two averagish hitters -- which polanco and kemp were this season -- could be separated due to differences in defense and base running.

Well, I think they're separated by about something like 0.85 WAR offensively overall (Kemp over Polanco), so that includes base running.

xp

I mean, the 0.85 number, as I mentioned, comes from runs created, which is a stat that seems to make sense but maybe even James didn't know why? I don't know why. At least it includes total bases so, presumably, base running.

when i say he saw less action, i'm referring to the fact that he played the same number of innings but had far fewer balls hit into his zone. (polanco played 1280 innings, not 1220, so they were essentially neck and neck.) this obviously was due to no fault of his own; likely it's due to the fact that his pitchers tend to allow more ground balls. this is a prime example of something that could be adjusted for, which would make polanco look even better in comparison

also, you might be underselling the value of a hit: http://www.insidethebook.com/ee/index.php/site/comments/run_values_of_events/

also, i don't think 10 runs created necessarily = 1 WAR. (this is obvious: kemp "created" 80 runs last season by james' (now-antiquated) method, but he certainly wasn't worth 8 wins on the offensive side.) 10 adjusted runs above average is roughly equal to that, but the two are different things

the way we're framing this makes it sound like polanco is some defensive wizard. he's not -- he's thoroughly average, both at the plate and in the field. hence his WAR total of about 2, which is that of an average player. kemp is (or was this year) a marginally above average hitter and an atrocious defender. polanco wasn't 25 runs "better" in the field so much as kemp was 25 runs worse

10 adjusted runs above average is roughly equal to that, but the two are different things

this should read above replacement, not average

also, i don't think 10 runs created necessarily = 1 WAR.

Neither do I, I was saying 10 runs more than another player, both above replacement level, would be 1 WAR.

I was using Polanco's numbers in right field only - 1220 innings.

the way we're framing this makes it sound like polanco is some defensive wizard. he's not

I had no intention of framing it this way at all. I was merely comparing their numbers.

10 "runs created" in the bill james sense also does not = 1.0 r/fWAR, but we're really missing the forest for the trees at this point. bottom line -- it's not hard to understand why an all-bat/no-glove guy can be less valuable than an all-around player, especially when the all-bat guy was roughly an average hitter

you might be underselling the value of a hit

The Dodgers this year had .496 runs for every hit, the Cardinals had .467, the Phillies had .456, and the Marlins had .432. Between the four teams, they averaged .462 runs per hit.

So, for 26 hits, that's about 12 runs and not 10.

I think the forest for the trees might actually be lost in that 0.6 WAR for a guy who was fourth in the league in RBIs but made 26 fewer OOZ putouts than Polanco.

you really love RBI huh? he was a barely above average hitter, played one of the least demanding positions on the field, and played it very poorly. even the jon heymans of the world can understand that that doesn't exactly scream "valuable player"

i also don't think it's a stretch to assume that the average hit outside an outfielder's zone is going to be worth a little more than the average overall hit

but look, if your aim here has just been about discrediting advanced fielding stats, i'm not gonna play anymore. i figured i'd at least attempt to help you understand

I think it's an interesting stat in that it involves actual runs, but I also think I understand all the caveats about the degree to which it's circumstantial.

My main point is that the discrepancy between the two players seems, to me at least, like a ton of mileage to account for given what we see with their offensive and defensive numbers (including the zone stats I brought up).

but i do think that skepticism that a case like this could actually happen is grounded in an old-fashioned belief that defense is of negligible importance

People always believed defense was important, but sayings like "defense never goes into a slump" used to be taken as gospel without any supporting evidence. Maybe it's more accurate to say that the value of good defense was well appreciated, at least qualitatively, but the negative value of bad defense was almost completely ignored.

i also don't think it's a stretch to assume that the average hit outside an outfielder's zone is going to be worth a little more than the average overall hit

Yes -- OOZ putouts are likely to be hit to the gaps and would go for extra bases if not caught. The average value of a single is around 0.4-0.5 runs, but I think a double is in the 0.7-0.8 range.

Does charging in on a ball and saving a single count as an in zone putout? If so, then a below average fielder might allow 10-20 balls to fall in front of him for singles per season, but "make up" for them by catching more routine fly balls due to his pitching staff.

What do you guys think of this assertion from the FanGraphs UZR Primer page:

"A typical outfield hit is worth around .56 runs and any batted ball out is worth around -.27 runs, so the difference between a hit and an out is worth around .83 runs."

The numbers I had last night showed that a hit for the Dodgers/Cardinals/Phillies/Marlins this year was worth about .46 runs. They're qualifying it by saying "outfield hits," but the number of infield hits expands that by .10? That would mean that infield hits make up 17.9 % of all hits.

And a hit for any of those four teams this year was a hit and not an out, so therefore "the difference between a hit and an out." I don't understand the premise of of subtracting another -.27 runs because it wasn't an out at all.

you should really take this up with someone with more expertise. suffice it to say that your back of the envelope calculations are less valid than the empirical data gathered by people who do this for a living

The numbers I had last night showed that a hit for the Dodgers/Cardinals/Phillies/Marlins this year was worth about .46 runs. They're qualifying it by saying "outfield hits," but the number of infield hits expands that by .10? That would mean that infield hits make up 17.9 % of all hits.

...no...

a hit into the outfield is going to do more damage than a hit through the infield. it's not a 1:1 relationship

but again, look, if it makes you feel better, you're right. kemp was the more valuable player because he had a lot of RBIs. defense doesn't matter that much. you win

I was skeptical about the defensive difference between Kemp and Polanco being worth something like 27.5 runs over the season based on the numbers. That is all.

this has been bothering me (although i love adam eaton)

http://www.fangraphs.com/blogs/instagraphs/adam-eatons-year-for-the-ages

doesn't really inspire me to believe in current defensive metrics tho

no one is comparing him to Clemente yet

have we all seen this?

https://baseballsavant.mlb.com/catch_probability_leaderboard

IsBrittonInYet:
When evaluating outfield defense, how much stock do you put in Statcast’s numbers (ex. outs above average, number of 5 star catches) compared to DRS/UZR? Kevin Pillar had a DRS/UZR of 15/6 in 2017 but was -2 outs above average with no 5 star catches in 34 chances.

Dave Cameron:
It’s not about “stock”; it’s about what the information is telling you. So, for instance, Pillar’s value could come from positioning, which would mean that he doesn’t make as many plays where he has to run a long way in a short amount of time, but he’s still catching balls other worse-positioned CFs wouldn’t get to.

Also, it’s important to keep in mind that Statcast is just measuring range right now, so you don’t want to compare that measure to total DRS/UZR, which include errors and throwing arm.

If you just look at the range portion of UZR, Pillar is at +1; his +6 came from throwing and not making many errors.

Colorado Rockies third baseman Nolan Arenado and Minnesota Twins center fielder Byron Buxton won the 2017 Rawlings Platinum Glove Award, presented by SABR. The winners were unveiled on Friday, November 10 during the 2017 Rawlings Gold Glove Award Ceremony, where the game's defensive stars accepted their hardware from some of baseball's greatest legends.

This is the first Rawlings Platinum Glove Award for both Arenado and Buxton. Arenado is the first infielder to start his career with five straight Rawlings Gold Glove Awards. Buxton is the first Twins center fielder to win a Rawlings Gold Glove Award since Torii Hunter in 2007.

The Rawlings Platinum Glove Award winners are determined by combining votes based on an adjusted SABR Defensive Index™ for each of the nine Rawlings Gold Glove Award winners in each League and the votes from baseball fans worldwide.

Based on the final SABR Defensive Index, Buxton was second in the American League with a 20.0 SDI, and Arenado was second in the National League with an 11.0 SDI.

I just skimmed this. The mere fact he's writing about baseball is encouraging enough--he mostly writes about the Cleveland Browns these days.

http://joeposnanski.com/dwar-to-end-all-wars/

(Love seeing Devon White's name on the CF list.)

1) I totally buy the idea that players can bring significant value via defense.

2) I have not seen every Cubs game this year, so maybe I'm missing something.

3) I stand by my years-old contention that publicly available defensive metrics are crap when this is what they tell me. pic.twitter.com/6yo6AuoWyV

— Jeff Passan (@JeffPassan) July 5, 2018

Pio:

DRS thinks Machado is a historically bad shortstop and UZR thinks he’s just regular bad. Which one do you think is closer to the truth?

Jay Jaffe:

In general, I tend to prefer DRS to UZR because of the additional observational input beyond just batted ball type (both of which do have their biases, admittedly). But I think a good strategy when viewing defensive metrics is to be wary of the outliers, and DRS tends to have more than UZR does — the spread from top to bottom is generally wider. So I’ll go with garden-variety bad instead of historically bad.

the response to passan about schwarber's defense is that he has average-ish range and a good arm; passan's response was that that's reasonable in a vacuum, but is he really a better defender than almora or heyward? because that's what the stats are saying