HEAD-TO-HEAD: Creating a new SP consistency metric

Introduction

It's late July of 2019. Your number one starting pitcher, Chris Sale (SP, BOS), has been largely inconsistent. You are desperate for strikeout and ratio help. You scan the waiver wire. There he is. The savior, Reynaldo Lopez (SP, CHW). He's just run off six very good starts. His ERA during those starts was 2.34 and his peripherals were excellent (10.6 Dom; 5.5 Cmd). Although his xERA of 3.87 showed that he was pitching slightly above his weight, he was the former #26 overall prospect according to BHQ in 2017. Perhaps he was finally putting it all together?

If you took the bait and added Lopez, then you know how this story ends. In August, his ERA was 5.34. In September, he essentially matched it with a 5.23 ERA. Simply put, Lopez could have single-handedly ruined your ratio categories down the stretch last year. He was one of the most inconsistent starters in the league, with 55% of his starts ending up as PQS-DISasters. The only team you would want him on is the team you were matched up against. In head-to-head play, it is critical to avoid rostering starting pitching that is inconsistent. This article will map out how to do that, and seek to quantify multiple forms of consistency in a single metric.

Step 1: Finding Pitchers Who Are Consistently DOMinant

The first characteristic we should look for is a pitcher who is consistently dominant on a start-to-start basis. Our beginning point for this analysis is to review those pitchers who achieve a high percentage of Pure Quality Starts (PQS). Pure Quality Starts are determined based on a five-point system that scores a pitcher's individual start based on innings pitched, hits, strikeouts, walks and home runs. A full explanation can be found here. A perfect PQS score is a 5. A start is considered PQS-DOMinant (DOM%) if it rates as a PQS-4 or PQS-5. A start is considered a PQS-DISaster if it rates as a PQS-0 or PQS-1.

Here were the 25 most dominant starting pitchers in 2019:

Name Tm Dom %
deGrom J. 
NYM
65.63
Manaea S.
OAK
60.00
Verlander J.
HOU
58.82
Glasnow T. 
TAM
58.33
Clevinger M.
CLE
57.14
Scherzer M. 
WAS
51.85
Strasburg S.
WAS
51.52
Kershaw C.
LAD
50.00
Yarbrough R.
TAM
50.00
Bieber S.
CLE
48.48
Morton C.
TAM
48.48
Bauer T.
CIN
47.06
Lynn L.
TEX
45.45
Greinke Z.
HOU
45.45
Wheeler Z.
PHI
45.16
Ryu H.
TOR
44.83
Buehler W. 
LAD
43.33
Cole G.
NYY
42.42
Flaherty J.
STL
42.42
Paddack C.
SD
42.31
Carrasco C.
CLE
41.67
Weaver L.
ARI
41.67
Syndergaard N. 
NYM
40.63
Civale A.
CLE
40.00
Peacock B.
HOU
40.00

We see some expected names on this list—Jacob deGrom (SP, NYM), Justin Verlander (SP, HOU), Clayton Kershaw (SP, LAD)—some potential mid-round values in Tyler Glasnow (SP, TB) and Charlie Morton (SP, TB), and some surprises resulting from a small sample size like Sean Manaea (SP, OAK) and Aaron Civale (SP, CLE). But this list is really only part of the consistency equation. While these pitchers led the league in the percentage of their starts that were PQS-4s or 5s, we don't know how susceptible they were to PQS-DISasters. The next step in the process is to incorporate DISaster (DIS%) ratings.

Step 2: Taking Note of Pitchers Susceptible to DISasters

By accounting for PQS-DISasters, we minimize week-to-week volatility in ERA and WHIP. Sure, you would have enjoyed 40% of Brad Peacock's (SP, HOU) starts where he was PQS-DOMinant, but you also would have to endured the other 40% of his starts that were PQS-DISasters. It's challenging to predict when those DISasters are coming, and it's best to mitigate that risk up front on draft day.

We are going to look at PQS-DISasters in several ways. First, let's adjust our list of the most dominant pitchers by accounting for their starts that were PQS-DISasters. For those of you familiar with quality-consistency scores (QC scores), you may recall that they incorporate both metrics to provide us with a convenient measurement of how consistent a player is on a week-to-week basis. The formula is as follows: (DOM% - (2 x DIS%)) x 2.

Interestingly, not all of the Top 25 most DOMinant starters were the most consistent as based on QC Scores. Does anyone have a guess as to which pitcher currently going in the 5th round of NFBC drafts doesn't qualify? Let's take a look at the pitchers with highest QC Scores in 2019:

Name           Tm      QC Score
==========     ===     =====
Verlander J.   HOU     94.12
Glasnow T.     TAM     83.34
deGrom J.      NYM     81.26
Clevinger M.   CLE     76.2
Scherzer M.    WAS     74.06
Strasburg S.   WAS     66.68
Severino L.    NYY     66.66
Ryu H.         TOR     62.06
Bieber S.      CLE     60.6
May D.         LAD     50.00
Kershaw C.     LAD     42.84
Lynn L.        TEX     42.42
Manaea S.      OAK     40.00
Cole G.        NYY     36.36
Flaherty J.    STL     36.36
Morton C.      TAM     36.36
Bauer T.       CIN     35.28
Giolito L.     CHW     34.50
Milone T.      SEA     33.34
Buehler W.     LAD     33.34
Woodruff B.    MIL     27.28
Gallen Z.      ARI     26.66
Gray S.        CIN     25.82
Boyd M.        DET     18.76
Syndergaard N. NYM     18.74

Those pitchers who were not listed on the Top 25 DOM% list are highlighted in yellow. And what PQS-DOMinant pitchers are missing once we control for PQS-Disasters? Early 5th round pick, Chris Paddack (SP, SD), as well as Zack Greinke (SP, HOU), Zack Wheeler (SP, PHI), Ryan Yarbrough (SP, TB), Carlos Carrasco (SP, CLE), Luke Weaver (SP, ARI), Aaron Civale (SP, CLE), and Brad Peacock (SP, HOU).

Not surprisingly, the common denominator among these pitchers is higher DISaster rates than the other pitchers listed in the Top 25 DOM% list. Their average DISaster rate was 26%, whereas the highlighted pitchers had an average DISaster rate of 7.98%. Even if we remove the two 0% PQS-DISasters from the highlighted list by correcting for small sample size in the case of Dustin May (SP, LAD), and complete fluke in the case of Tommy Milone (SP, SEA), the DISaster rate of the highlighted group is still only 11.17%. We also note that Paddack's DISaster rate was somewhat abnormal due to innings management and maintenance down the stretch last year. He started off the year avoiding a PQS-DISaster in his first seven starts. It would not surprise us to see him on this list next year, and as you will see below, he receives an endorsement on our aggregate consistency ratings.

And what about some of the highlighted newcomers? Brandon Woodruff (SP, MIL), Zac Gallen (SP, ARI), and S. Gray (SP, CIN) all offer solid floors with upside at ADPs ranging from 91-129. They each have shown the ability to minimize blow-ups and are sound mid-round targets.

Now, let's look at PQS-DISasters from another lens. Who are the pitchers with the highest 2020 ADPs that had DIS%s over 20%?

NAME              TM     ADP   DIS %
==============   ====   ====   =====
Snell B.         TAM     38    47.83
Corbin P.        WAS     49    27.27
Nola A.          PHI     51    26.47
Kluber C.        TEX     89    28.57
Berrios J.       MIN     93    25.00
Paxton J.        NYY     93    31.03
Carrasco C.      CLE     109   25.00
Rodriguez E.     BOS     131   20.59
Fried M.         ATL     146   23.33
Hendricks K.     CHC     161   26.67
Urias J.         LAD     163   25.00
Manaea S.        OAK     170   20.00
Minor M.         TEX     172   25.00
Odorizzi J.      MIN     180   23.33
Heaney A.        LAA     184   27.78
Price D.         BOS     185   22.73
Weaver L.        ARI     186   25.00
Foltynewicz M.   ATL     200   28.57
Musgrove J.      PIT     214   25.81
Smith C.         MIA     218   35.71
Lucchesi J.      SD      219   33.33
Stroman M.       NYM     220   28.13
Canning G.       LAA     223   29.41

With the exception of Sean Manaea (SP, OAK), who only started 5 games last year, all of these pitchers had negative QC Scores in 2019. While some of these pitchers generated fairly good overall results, it was a bit of a bumpy ride to get there. The lesson here is that if you are investing a mid-to-high-round pick on a starting pitcher, be sure to review their DIS% in isolation to understand the frequency of their pitching DISasters.

Step 3: Factoring in Health

In 2019, we saw starting pitching drafted earlier than ever, only to see multiple aces miss significant time due to injuries. We would be remiss not to take health into account. Although our final analysis below will use IL days over the past three years, when we review the Health grades of the pitchers with the Top 25 QC Scores, the results are eye-opening. 13 of the 25 pitchers received grades of D or F. Only 5 pitchers with more than one years' experience received a health grade of A. These grades highlight another layer of consistency—the ability to consistently make it to the mound every fifth day. It's one thing to be consistent from start-to-start, but a pitcher also needs to be able to consistently make those starts. We will incorporate health as a consistency component in our final analysis.

Step 4: Putting It All Together into a Single Consistency Metric

As we just saw, there is an element of each component that is important in insolation. But if we combine these variables into one unique score, we get a much more complete picture. We have created an aggregate consistency score (ACS) that combines these factors and rates each pitcher on a consistency scale of 20 to 100. For each of (1) DOM%, (2) DIS%, (3) QC Score, and (4) days on the IL the past 3 years, each player is assigned a score of 1 to 5 points. That score is then multiplied by a factor of 5 in order to curve the scores to 100.

In order to calculate each player's score per the four metrics listed above, their performance in those metrics was assigned to certain percentiles:

DOM%                 DIS%                    QC Score             IL-Last 3 Years
==================   =====================   ==================   ==================        
Top 20% = 5 points   Lowest 20% = 5 points   Top 20% = 5 points   Top 20% = 5 points
2nd 20% = 4 points   2nd lowest = 4 points   2nd 20% = 4 points   2nd 20% = 4 points
3rd 20% = 3 points   3rd lowest = 3 points   3rd 20% = 3 points   3rd 20% = 3 points
4th 20% = 2 points   4th lowest = 2 points   4th 20% = 2 points   4th 20% = 2 points
5th 20% = 1 point    5th lowest = 1 point    5th 20% = 1 point    5th 20% = 1 point

Note: for the IL days, the "Top 20%" refers to those pitchers with the fewest number of IL days across the past 3 years.

Here are the Top 60 starting pitchers based on their ACS. Also, please note any pitcher with a DOM% of 0% was not included:

Name            Tm    ADP   ACS
==============  ===   ===   ===
Verlander J.    HOU   14    100
Bieber S.       CLE   29    100
Lynn L.         TEX   134   100
Cole G.         NYY   6     100
Flaherty J.     STL   25    100
Boyd M.         DET   166   100
deGrom J.       NYM   9     95
Greinke Z.      HOU   69    95
Paddack C.      SD    52    95
Montas F.       OAK   123   95
Gallen Z.       ARI   129   95
Scherzer M.     WAS   18    90
Yarbrough R.    TAM   261   90
Morton C.       TAM   61    90
Bauer T.        CIN   83    90
Buehler W.      LAD   18    90
Gray S.         CIN   102   90
Giolito L.      CHW   55    90
Castillo L.     CIN   51    90
May D.          LAD   248   90
Glasnow T.      TAM   69    85
Clevinger M.    CLE   24    85
Strasburg S.    WAS   30    85
Kershaw C.      LAD   56    85
Corbin P.       WAS   49    85
Urquidy J.      HOU   232   85
Minor M.        TEX   172   85
German D.       NYY   236   85
Wheeler Z.      PHI   124   80
Ryu H.          TOR   106   80
Syndergaard N.  NYM   75    80
Soroka M.       ATL   105   80
Sale C.         BOS   39    80
Woodruff B.     MIL   91    80
Voth A.         WAS   472   80
Berrios J.      MIN   93    80
Smeltzer D.     MIN   604   80
Manaea S.       OAK   170   75
Carrasco C.     CLE   109   75
Weaver L.       ARI   186   75
Civale A.       CLE   288   75
Musgrove J.     PIT   214   75
Canning G.      LAA   223   75
Severino L.     NYY   60    75
Odorizzi J.     MIN   180   75
Nola A.         PHI   51    75
Darvish Y.      CHC   71    75
Marquez G.      COL   176   75
Maeda K.        LAD   202   75
Teheran J.      LAA   354   75
Plesac Z.       CLE   362   75
Urias J.        LAD   163   70
Bumgarner M.    ARI   116   70
Paxton J.       NYY   93    70
Chirinos Y.     TAM   288   70
Lopez P.        MIA   340   70
Hendricks K.    CHC   161   70
Rodon C.        CHW   740   70
Gonzales M.     SEA   358   70
Stroman M.      NYM   220   70

We stress that these are not intended to be used as straight rankings. Instead, they should form a component of your own rankings. We are providing you with a spreadsheet that contains the consistency data used in creating this article, as well as BHQ's 2020 projections. We encourage you to run your own filters based on your league's parameters and your own targets. By using filters like projected Cmd and xERA you can further refine your target pool of pitchers.

Conclusion

The quest to identify continuous consistency is always ongoing. Pitchers can improve. They can change pitch mixes. They can learn new pitches. They may find a new arm slot. They can also can get worse. They age. Their velocity can decline. They can get hurt. Famed British author Aldous Huxley once said, "Consistency is contrary to nature, contrary to life. The only completely consistent people are dead." With that sobering thought in mind, we should note that despite our best efforts to identify the components of consistency and quantifying it, there will always be a level of imperfection.

But it is that imperfection that keeps us engaged during the season. On draft day, our perspective and knowledge is different. We have historical results, our 2020 projections, and now, a new metric that we can use to build consistency into our rosters. If you leave your draft feeling confident that you have made pitching consistency a focus of your roster, then you have won the day. You have done all you can. Then, it is time for that long, fun six-month ride to begin...


Click here to subscribe

  For more information about the terms used in this article, see our Glossary Primer.