Rotoman’s 2016 Projections: How’d He Do?

I used to evaluate my projections each year against what really happened. When I started doing this, 22 years ago, I was ambitious and driven to get better results, but after many years the limits of our predictive ability are obvious. The bottom line is that there is no way to predictively model the game’s stochastic nature. Random stuff causes a significant percentage of the events that happen on the baseball field, and trying to guess whether those random events are going to go one way or the other is absurd.

Or rather, trying to guess them accurately is absurd. We continue to make our guesses, and we (and I mean all baseball predictors) continue to get something like 75 percent accuracy (which is a 25 percent failure rate).

That significant variance also makes it hard to judge whether improvements are actually improvements or not. If I score around 75 percent each year, does it mean something systematically when that number dips to 73 percent, or bumps up to 77 percent, the next year? Or is that a reflection of randomness? At the very least it’s hard to tell.

But I didn’t stop measuring because it’s hard to tell. I stopped because the timing was bad. The season ends and I’m all in on the Guide, there isn’t much time to do the comparison, and not much urgency given how little there is to be learned.

When the Guide is done, it’s the holidays, and then a rush to complete the much more extensive projections in the Patton $ Software by the end of January. And once that’s done, there’s getting ready for the season. But this year, someone asked specifically for an evaluation, so I put together a spreadsheet. You can see it here.

The first thing I check is the overall accuracy of the Top 100 hitters and pitchers. That is, I look at the Top 100 hitters projected for the most at bats, and compare their category totals with what actually happened. This year:

AB: 94 percent
R: 99 percent
2B: 93 percent
3B: 90 percent
HR: 114 percent
RBI: 100 percent
BB: 101 percent
K: 96 percent
SB: 86 percent
CS: 85 percent
BA: 101 percent

Not perfect, and not necessarily imperfect in obvious ways.

One of the standard ways to measure the accuracy of projections is to use the Correlation Coefficient, which measures the extent to which two variables (in this case the 2016 projection and the 2016 actual result) have a linear relationship. To be a little more brass tacks about it, a correlation of 1 means that two sets of data create the same angle when graphed, even if they show up in different parts of the graph. 3, 4, 5 would have a correlation of 1 with 5, 6, 7.

A correlation of 0 means that the two data sets are completely unrelated to each other.

Most interestingly, a -1 correlation would mean that the second data set would be at a 90 degree angle to the first. Negatively correlated.

With that in mind, here are the correlations for my 2016 projections compared to what actually happened.

Screenshot 2017-02-03 23.44.55

The first thing to note, for my self esteem, is that when we look at the Top 500 projected hitters, I hit the 75 mark in AB, R, HR, RBI and, almost, SB. That’s the holy grail, I think. You want your set to reach .75 in correlation. That’s a pretty good correlation, if you know what I mean.

But, and big but, the numbers are much more problematic when measuring the Top 100 projected hitters. AB is a mess, but oddly HR, RBI and SB aren’t that bad. Remember that .75 is about as good as it gets, though that statement comes with provisos.

What I’m getting at here is that there are many ways to evaluate projections.

If you look at the whole data set, as we do here in the Top 500 projections, we get about the results we hope for. This is the limit of a baseball projection, or close to it.

Another way to evaluate projections is to sort by the actual number of at bats players actually had. This gives you a list of the most active players on the year, and how best we predicted that.

Screenshot 2017-02-04 00.10.22

A little better, it turns out, which means that we’re doing better predicting who actually plays and how they produce than we are predicting what the most predicted guys are going to produce. By a little.

Pitchers are going to have to wait for later, but I hope this gives a little bit of a taste about what projection reviewing means. Maybe we’ll take a look at some other systems, too, coming up.

Corrections for The Fantasy Baseball Guide 2017

fbg2017-cover-largeFirst off, if you want to read the Mock Draft Commentaries, go here.

If you would like the FBG projections and prices update, it is here. The password is the last name of the first player profiled on page 90 of the 2017 Fantasy Baseball Guide. It is case sensitive.

This is the place where I’ll post corrections and updates to the 2017 Guide. There is a link in the top nav bar, so you can always find it.

Page 6:  Talking about team names, as we do sometimes, Yoan Moncada and Lucas Giolito were traded from Boston and Washington respectively last winter. While the team changes for both, to the White Sox, made it into their capsules, the wrong team names linger on this page of rookies.

Page 6: Very embarrassing. The photo credit is wrong. The picture of Yoan Moncada is by friend and new contributor Buck Davidson. I’m sooo sorry Buck, and will get a fixed PDF for you to use as a clip.

A Reader Writes:

The Fantasy Baseball Guide is a great pub and is most useful.  I have one question and one suggestion:

Q. It appears to me that Matt Moore’s “Big Price” of $14 is not consistent with his projected stats of 4.44 ERA and 1.35 Whip.   I believe his Win and K totals of 11 W and 127 K are very average.  I understand he has upside potential, but it appears too me that the $14 projection is not supported by your projections. Please explain.

S. It seems to me it would take little effort on your part to include the player’s team either in the “Player’s by Position” section or the Full Profiles.  I play in a hybrid NL + 2AL teams (Houston and Texas) and many of us have limited knowledge of other AL players.  I understand there will be changes between the start of the season and your publication date, but that’s not near the problem of having no idea of the player’s team (and whether or not he is on one of our teams).  Please consider including the team for each player.

Thank you for your consideration.

Dear Reader. The projection in the Guide was the mechanical projection that derived from Moore’s history, and as you note looked rather pessimistic for a player I’m fairly keen on and three other writers made PICKS of this year. I work on the projections all winter and at some point I upgraded Moore to a 3.72 ERA and 1.23 WHIP. It just so happens this projection, also in 150 innings pitched, is worth about $12 and I’ve dropped his bid price to $13. Note that in CBS and LABR, Moore went for $13 and $11, so I was definitely in the ballpark on Moore’s bid price if not his projection in the Guide.

The Guide projections and price update is due tomorrow, probably in the evening, here on the corrections page. It will have hugely reworked projections and bid prices for those who bought the Guide. You may still find it on the shelves at Barnes and Noble and other magazine retailers, and you can also buy the online version at thefantasysportsguide.com. Use the promo code Rotoman17 and get $1 off.

As to the issue of team names, they are available in the stat lines for all the players with major league experience last year. And I do add team names to the end of the prospect capsules because readers very much want them, but it really is a problem to be more definitive. The magazine heads off to the printer just days after the Winter Meetings conclude, and at that point there are still hundreds of free agents out there. Plus, trades will be made. So, the choices are to either list the team name that the player ended last year with, which in hundreds of cases will be wrong, or their team name at press time, which means hundreds will be listed as a Free Agent.

It’s always seemed to me that the team name in the stats is just as reliable as either of the above methods, and doesn’t pretend to an authority that we don’t have in mid December. I’m open to suggestions, surely, so please feel free to send them along.

A reader named Jeffrey reports: I figure that if Tyler Anderson and Jon Gray are worth $2 apiece, then Tyler Chatwood should be worth a bid of $1. If I see anything else of note, I will send another email.

Read more…

Ask Rotoman: Rank the Cardinals Outfield

Rotoman:

I’m in a head to head points league. Should I hold on to Randal Grichuk or would one of the other Cardinals OF be a better option.

“Jack of Hearts”

Matt Holliday is the best St. Louis outfielder, but he comes with the dreadnought of age and some injury history.

I think Grichuk is a smidge more valuable than Piscotty, but what really matters is whether you value BA or HR more. Grichuk has more power but will hurt your batting average, while Piscotty isn’t going to hit as many out but should have a better batting average.

You can tell best what your team needs. But if you can get Matt Holliday, go for it.

Sincerely,
rotomansignature

The Fantasy Baseball Guide 2016 in Brooklyn!

I spied the Guide in the wild today, at the Park Slope Barnes and Noble. Just us and the Sporting News amidst the gun mags.

2016-01-11 14.40.27
Guitar mags, too.

Gushy story about CC Sabathia

It’s in the NY Times and makes Brian Cashman sympathetic, too.

I think the timing here has to be selfless. CC needed help, and he said he did.

 

 

2014 Doubt Wars Results are Released

doubtwarslogo-150x150Glenn Colton and Rick Wolf wowed us by finishing first overall for the second year in a row, and this time actually won the Tout AL title, too, but the real eye opener was Bob Russo’s Triplets, which won the Doubt Wars mixed title with 23 points. He finished first of 61 teams in eight categories!

Get all the results at Toutwars.com: AL, NL, Mixed.

Ask Rotoman: Four For the Price of One?

Dear Rotoman:

I am pretty sure I already know the answer, but I have a question about who to start in my lineup for week 22.  I have Chris Sale and Max Scherzer and they are pitching against each other.  I could sub one of them for a 2 start Chris Tillman or Kyle Lohse. Should I do it?

“Doubling Up or Down?”

Dear Doubling:

Sale and Scherzer are better than Tillman or Lohse, so the answer generally is to play them.

But if you’re in a cluster of wins in the standings and they have outsized importance down the stretch, it would make sense to get extra starts. Tillman has been very effective in recent months and would make a gutsy and bold move that could pay off.

If your ERA and WHIP are safe, it may make sense to play both Tillman and Lohse instead of your two aces facing off against each other, since you’ll

But don’t you have less effective pitchers who have one start you could replace instead?

Slicingly,
Rotoman

On Sun, Aug 24, 2014 at 4:52 PM, Paul <wordpress@www.askrotoman.com> wrote:

Name: Paul

Email: vintage1970@aol.com

Ask Rotoman: I am pretty sure I already know the answer, but I have a question about who to start in my lineup for week 22.  I have Sale and Scherzer and they are pitching against each other.  I could sub one of them for a 2 start Tillman or Lohse.

Jason Collette, Monitoring the Small Sample

Jason has an excellent post over at Rotowire, called Monitoring the Small Sample, that makes the obvious but excellent point that:

“All change has a starting point. If a pitcher’s strikeout rate needs 150 batters faced to become meaningful, it does not mean we ignore plate appearances 1 to 149. Rather, it means we watch the player’s process during that span to see if something has changed or if the improved numbers are another example of statistical randomness. Last season, Fernando Rodney faced just 42 batters in April but had seven saves and a 0.87 ERA.”

And what fantasy player wasn’t buying Rodney as soon as they possibly could, despite the universal belief that he would self destruct at any moment? Every fantasy player was buying. Not always happily but out of necessity.

But if we know certain things about a player’s or team’s approach, maybe that helps identify one of those starting points. Jason says:

April served as a starting point for the changes Rodney made in his delivery and his position on the rubber that led to better short-term results that eventually became better long-term results. Discussing, tweeting, or writing about such changes to deliveries, positioning of hands and feet, or swings is not confirmation bias of statistical recency as much as it is looking for starting points for change.

The bottom line for fantasy players in deep leagues is that we don’t have time for confirmation. If there is a whisper of hope to improve we jump on it, not always prudentially. But in shallower leagues the questions are different. Knowing that small sample success might be fueled by a change in approach or situation might get you out ahead of the crowd.

Note: Rotowire is a pay service and if you’re not a subscriber I don’t know what the link will get you. I pay for Rotowire and wholeheartedly recommend it.

Outliers: Finding Disagreement With Myself

Creating my initial Bids and my initial Projections are two discrete processes.

For the Bids I sit with a player’s history of Cost and Earnings, look at his age and any injury information, and try to determine how much I think he’s worth and how much I think everyone else thinks he’ll be worth, since everyone else is who I’ll be bidding against. If I’m way higher than I think the market will be, I’ll shave my bid down so that it will win, but not tower above the competition. And if I’m way lower than I think the market will be, I’ll bump my bid up to just below the market. I want to indicate my predilections, but I’m also trying to describe the market as a whole, so the prices are useful even if you disagree with me.

My projections come in two phases. The first is running a player’s historical data, including a bunch of component parts (batted ball data, mostly), through my projection formula, which also takes into account age and league and home field and league. This gives a rough idea of what players will do but has to be adjusted for playing time, and for changes in roles. After those adjustments the projections run in the Guide, and a similar but more complex set (more categories, mostly, and more time to smooth anomalies) run in the Patton $ software, at first. As spring training progresses I tweak the projections manually, mostly for playing time with veterans, but also to deal with differing situations on teams with platoons and competition and as I get a better sense of new players and their roles.

Once the projections are loaded into the Patton $ software they get priced using Alex’s formula, which is an excellent way to discover what the projection formula is telling me, especially when it differs substantially from the bid price. I’ve been going through the lists, looking at some of the substantial differences, assuming that these are players who might be of special interest this year.

HITTERS (Proj$, PK5)

Mike Trout ($49, $41): The bid predated the reports about Trout’s reporting weight. It assumes he’s not going to be nearly as good as last year, but still plenty good. THe projection is a result of increased playing time, even though he’s projected to not be nearly as good as last year. Verdict: Assuming he can run once the season starts, I would be fine standing by the projection, but I think there’s enough risk of sophomore slump and/or other issues that I wouldn’t bid more than $41.

Albert Pujols ($36, $31): The projection is remembering Albert’s past greatness. Age deductions of significance don’t kick in until the mid 30s. He could be great again, but the trend is clear. Verdict: One reason to bid $31 on Pujols is that he could put up another $36 year. But counting on an aging player to keep running is a mistake. I’ve bumped his projection down a bit, especially the SB.

PITCHERS (Proj$, PK5)

Joaquin Benoit ($15, $1) The bid is wrong. It is the standard bid for a setup guy in 5×5. Especially a guy on a team with a different pitcher named as closer (Bruce Rondon) and at least two other worthy CIW candidates (Al Albuquerque and Phil Coke). But it’s wrong because I’m projecting Benoit to be the closer at some point this year, and to do a good job at it. So, I’m bumping him to $3. That may seem silly, but that’s what he’s worth if Rondon does the job (I doubt it) or one of the other guy ends up the closer. Verdict: Right now Benoit is a closer in waiting. Maybe not even first on line. The reason closers in waiting are valuable is because you don’t pay much for them. So until there’s more smoke, I’m going to keep the bushel on this fire.

Andy Pettitte ($12, $1) He’s 41 years old this year. He only pitched 79 innings last year, and took the year before that off. It’s fine to say last year’s injury was not age related, but not many pitchers stay effective and healthy into their 40s. The projection reflects what he might do if he stays healthy, but the bid is a severe hedge. Verdict: It will be in the $8-$10 range if he emerges from ST in the rotation.

More to come!

The Fantasy Baseball Guide 2012: Coming February 15th

This year’s guide is at the printer, and should start showing up in stores soon. Here’s the front cover.