RESEARCH: ERA/xERA outliers

Here at BaseballHQ.com, we like to embrace precision. At the same time, we understand the nature of any imprecision. A recent look at xERA, though, revealed some oddities. As part of the formulation process, we use a factor to adjust xERA such that league-wide, ERA and xERA agree. This makes sense, as there are likely factors beyond those in the xERA calculation that influence ERA.

However, if the xERA calculation was unbiased, we would expect the factor to be a random walk, sometimes above zero and sometimes below. In fact, it has gone in the same direction for the past 13 years.

So this doesn't look at all random. In fact, in each year for the past 13 years, the factor has been positive (i.e., calculated league xERA has been higher than actual league ERA). The odds are about 1 in 8,000 of this happening randomly, so there definitely appears to be some bias.

Well, we're not going to tear apart the xERA formula here. What we will do, though, is examine the different inputs to xERA to see if we can find any patterns. These patterns can help us better analyze large xERA/ERA variances.

As a reminder, here is the formula for xERA:

xER = xER% * (FB/10) + (1-xS%) * (0.3 * BIP - FB/10 + BB)

where

xER% = 0.96 - 0.0284 * G/F

xS% = [64.5 + K/9*1.2 - (BB/9^2 * + BB/9) / 20 + 0.0012 * GB%^2 - 0.001 * GB% - 2.4 ]/100

Trajectory

A logical place to start is trajectory. Do extreme fly-ball or ground-ball pitchers tend to blow up the model?

This is a promising start. It definitely seems that pitchers at the extremes (FB%<19% or FB>42%) exhibit some significant variances. And with few exceptions, the further from the center one gets, the greater the variance. The overall "U" shape of the curve suggests that xER may be a function of (FB/10)2 rather than just FB/10.

When we look at GB%, there is again more activity with the outliers. However, there is also a clear upward trend as GB% increases. Note that GB2 is already a component of xERA, so we can't necessarily conclude that the issue is the exponent. One thing we can conclude, however, is that we need to be more careful when evaluating ERA outliers among extreme ground-ball pitchers.

Dominance and Control

The other primary factors in xERA are dominance and control.

While there is a trend, it's rather all over the place until you get to an 8.5 Dom. From there, it's very clear that xERA isn't properly capturing the effects of Dom. We've seen a significant increase in Dom in recent years, so perhaps we need to revisit that the next time we look at xERA. Here again, expect greater variance between ERA and xERA for high-Dom pitchers.

As with Dom, better control means more variance with ERA. At this point, we've seen positive trends with control, dominance, and ground-ball rate. Fly-ball rate also showed more variance at the extreme lows, but also showed more variance at the extreme highs. Maybe the best pitchers are the ones who confound xERA. Is it possible that xERA is underestimating the impact of their skills? Let's find out.

Eureka! This is about as clear as it gets: for higher-skilled pitchers (say, those above 110 BPV), xERA isn't quite capturing their awesomeness. Note that a 15% variation for a guy with a 3.00 ERA is 0.45, which leaves a guy in probably the same overall class, even though it feels like a big difference.

Survivor Bias

Another possible source of variance is survivor bias: pitchers who underperform their xERA tend to lose their jobs or see their innings cut back, so they don't get the chance to regress to their true level. Pitchers who outperform xERA get more innings or get to keep their jobs. If that's the case, we would expect to see significant variation among pitchers with fewer innings pitched and less variation as innings increased. What we found was the exact opposite; variation increased with playing time. This is most likely due to the BPV effects we identified earlier. Simply put, better pitchers over time will get more innings.

Conclusions

Certainly, revisiting xERA at some point is something we will consider. In the meantime, we can use what we've learned here. Your best course of action is to have a higher tolerance when looking at xERA/ERA variation with high-skilled pitchers. A very general rule of thumb would be to consider variances of less than half a run to be statistically equivalent to zero. That won't always work (which is why it's a rule of thumb and not something more definitive), but it will likely yield better results than "selling high" on a high-skill pitcher who's outperforming xERA.


Click here to subscribe

  For more information about the terms used in this article, see our Glossary Primer.