Mike Sando's 2015 QB Tier Rankings, in which he polled 35 NFL insiders for their opinions on the league's 32 starting quarterbacks, had the Oakland Raiders' Derek Carr tied for 20th overall. That was on par with Jay Cutler and ahead of Nick Foles and Sam Bradford. On the surface, it's easy to understand the optimism surrounding Carr as he enters his second season. After a slow start, he led the Raiders to three wins in their last six games. Perhaps more importantly, he seemed to show up biggest in the most important moments: His red zone performance was the best in the NFL, according to Football Outsiders metrics; his Total QBR was at its highest in the fourth quarter; and his 21/12 TD/INT ratio was better than Andrew Luck's rookie-year mark (23/18) three years ago.
So what's not to like? Carr appears to be a cautionary tale about focusing on the wrong numbers. While Carr and Luck posted almost identical rookie years by traditional passer rating (Carr 76.6, Luck 76.5), their QBRs were quite different (Carr 38.4, Luck 65.2). And forget about Carr getting better as the season progressed: During the Raiders' 3-3 stretch to end the season, he posted a minus-16.6 percent mark in Football Outsiders' defense-adjusted value over average (DVOA) metric (explained further here). That was worse than his performance during the Raiders' 0-10 start to the year (-13.8 percent).
Using the right numbers, it's clear the odds are against Carr becoming a middle-of-the-pack quarterback.
Rookie seasons are telling
There have been 18 quarterbacks drafted outside the top 10 since 2000 who threw at least 200 passes as rookies. Carr ranks ninth in this group -- right in the middle -- according to DVOA. That sounds promising until you realize that most quarterbacks drafted outside of the top 10 fail in the NFL. The ones who have succeeded were generally pretty good as rookies.
In terms of the total value that quarterbacks generate in Years 3-5 of their career, measured by defense-adjusted yards above replacement (DYAR), each of the best four QBs -- Ben Roethlisberger, Russell Wilson, Andy Dalton and Joe Flacco -- ranked in the top five in rookie performance (see table). The quarterbacks who eventually succeed from outside the top-10 picks have not always been above average as rookies (Flacco), but none have been as bad as Carr.
This trend is not limited to just these 18 quarterbacks, either. If we expand the pool to the 31 rookie starters drafted outside the top 10 who threw at least 100 passes since 1989, rookie performance explains most (53 percent) of a quarterback's total value in Years 3-5. That number jumps to 61 percent since 2000. Rookie seasons, particularly recently, often say a lot about how a quarterback's career will turn out, which isn't great news for Carr.
Red zone randomness
Carr dropped back to throw 617 times as a rookie -- 52 of those came inside the red zone, with the remaining 565 coming outside the 20. And while his red zone DVOA was a league-best 57.1 percent, that stands in stark contrast to Carr's performance on all other throws: minus-22.8 percent DVOA (34th out of 36 qualifying QBs). Unfortunately for Carr, those struggles tell us far more about his future than his red zone success.
Forget where Carr threw the passes for a second. Fifty-two dropbacks equates to less than two games, a sample over which the worst quarterback in the 2015 QB Tier Rankings (Geno Smith) posted big numbers at the end of last season. Indeed, when you're dealing with so few attempts, success can reflect luck/randomness much more than skill.
And if you're thinking Carr possesses that ever-elusive clutch gene, consider: Red zone success bounces all over the place from year to year. Looking just at the past two seasons, red zone DVOA in 2013 explained just 12 percent of red zone DVOA in 2014 (minimum 30 passes each year). This fact alone means that we would expect Carr's red zone performance to fall back almost all the way to average in 2015. Factor in Carr's poor performance on all his other passes, and he projects to be below average in the red zone.
Cheap touchdowns
Then there is Carr's gaudy-for-a-rookie TD/INT ratio, slightly better than even Roethlisberger's 17/11 output in his great 2004 rookie campaign. This too appears to be a mirage: Carr's touchdowns were often of the gimme variety -- very short passes when other teams might have run it in.
For the season, Carr's touchdown passes traveled an average of just 6.5 yards through the air, compared to a league average of 11.9 yards. In fact, roughly half of Carr's TDs (11 of 21) came on throws within three yards of the line, while only 31 percent of all other quarterbacks' scores came on such passes.
The picture doesn't get much prettier when comparing Carr to the other rookie quarterbacks since 2012 who threw at least 10 touchdowns.
If we adjusted for degree of difficulty, Carr's touchdown total would be toward the bottom of this list -- a common refrain of his rookie season.