Erik Karlsson Hasn’t Turned Back the Clock. He’s Just Riding the Percentages.

Why I’m Not Fooled by the Blueliner’s First Four Games.

The San Jose Sharks have started off the season 4–0–0. They’re one of only six teams who remain undefeated, and one of only three teams who’ve won every single game in regulation thus far. For a team that earned a regulation win in less than 30% of the games they played over the previous two regular seasons, this is certainly a surprise.

At the forefront of this surprise is Erik Karlsson, San Jose’s highest paid player. The Swedish defenseman is set to earn a total salary of $14,500,000 this year, in the third of an 8-year contract which pays him $11,500,000 annually per CapFriendly. Karlsson has earned a point in each of San Jose’s games this season, and his 6 total points rank 2nd among defensemen, behind only Drew Doughty of the Los Angeles Kings. His point totals have led many to state that the Karlsson of old, a dominant offensive defenseman who won two Norris Trophies with the Ottawa Senators, is back.

Personally, I don’t see it that way. I actually believe Karlsson’s first four games have been more of the same from last season, where he finished near the bottom of the league by my Wins Above Replacement model, with estimated contributions well below those of a replacement player. After thoroughly reviewing the data and the film, I’m confident in my belief that Erik Karlsson’s impact on the game so far this season has been closer to what it was last season — very poor — than to what it was at his peak — elite. I’m also rather confident that it will continue to be closer to the very bad end of that spectrum going forward.

Breaking Down the Tape

While watching Erik Karlsson’s game last season, I noticed some weaknesses in his game that were either not present, or at least not nearly as problematic in his first season as a Shark:

1. Successfully challenging the flow of play. The success rate of his challenges is down considerably, and he also seems to challenge less often.
2. Defending opponents who attack with speed. This was often done through challenging the flow of play.
3. Taking efficient skating paths.
4. Managing the puck.

I believe the new-found appearance or exacerbation of these weaknesses are what have led to Karlsson’s very poor on-ice results, and I am confident they will continue to drive poor results over any meaningful sample size unless they are cleaned up or more effectively managed. Throughout the first four games of this season, I haven’t seen evidence of these things happening.

In the video below, I’ve clipped some plays from this season which highlight these weaknesses.

Erik Karlsson Today

Here’s what I see in this video:

• 0:00 — Karlsson reads a poor pass from WPG27 Ehlers and successfully challenges the flow of play immediately, but fails to manage the puck, creating a dangerous 2-on-1 for Winnipeg.
• 0:09 — MTL27 Romanov’s pass hits MTL40 Armia in the neutral zone and bounces off the boards, at which point Karlsson challenges the flow of play and skates forward towards the puck. MTL13 easily pokes it past him and creates a partial 2-on-1 in which Armia gets a solid scoring chance off.
• 0:17 — OTT2 Zub passes the puck in the neutral zone to OTT7 Tkachuk and Karlsson challenges the flow of play immediately as the pass is made, skating directly towards Tkachuk, but fails to make life any more difficult for him. Tkachuk easily passes the puck to OTT19 Batherson. Karlsson is now caught high on a dangerous 2-on-1 that he directly converted from a 3-on-2, and Batherson scores.
• 0:24 — WPG80 Pierre-Luc Dubois attacks with speed and catches Karlsson flat footed with just a slight change of direction as their paths cross; Karlsson’s stick never comes close to the puck in his futile poke check attempt, and Dubois releases a dangerous shot.
• 0:28 — MTL58 Savard generates a rebound with a shot off SJ33 Hill’s blocker while MTL28 Dvorak easily blows past Karlsson and attacks the rebound with speed, firing a dangerous one-timer at Hill. Dvorak’s one-timer generates a loose rebound which Karlsson fails to manage, and one which Montreal ultimately turns into another scoring chance on a deflection from Dvorak.
• 0:38 — Karlsson fails to manage the puck in the offensive zone, puts it directly into the skates of OTT10 Formenton, and then challenges Formenton’s new-found momentum with the puck. Formenton easily chips the puck off Karlsson’s skate back onto his own stick before getting off to the races, leaving Karlsson in the dust and creating a dangerous 2-on-1.
• 0:48 — Karlsson recovers a dump into the neutral zone with time and space to make a play, but fails to manage the puck and puts a pizza directly on the stick of attacking forward TOR25 Ondrej Kase, which ultimately creates a solid scoring chance.
• 0:53 — Karlsson takes an inefficient skating path in the neutral zone, shifting the majority of his weight in the direction of his own goaltender as Toronto enters the zone with speed before quickly pivoting his weight back in the direction of TOR47 Engvall as Engvall receives a pass just across the blue line. This inefficient path affords Engvall the chance to split Karlsson and S.J’s other defender and generate a dangerous chance which Karlsson, to his credit, reduces the danger of with a last ditch poke check.
• 0:57 — Karlsson rushes the puck out of his defensive zone with speed, but fails to manage the puck and has his pocket picked by TOR16 Marner who creates a dangerous scoring chance.
• 1:06 — TOR16 Marner passes the puck behind the net to TOR88 Nylander. Karlsson briefly takes an inefficient skating path in front of his own goaltender as the pass is made before turning back and putting too little pressure Nylander on too late, which ultimately leads to a dangerous deflection chance for Marner. Karlsson beats Matthews to the rebound, but fails to manage the puck and puts it back on the stick of Nylander. The puck is temporarily cleared, but TOR eventually recovers it in the neutral zone, and after TOR34 Matthews passes the puck to Nylander after entering the offensive zone, he attacks the net with speed, blowing right by Karlsson in the process. Nylander missed the net with a shot that was scoring chance in and of itself, but had he passed the puck back to Matthews or generated a rebound with the shot, this could have been a lot worse.

In order to make a proper comparison from what I saw last season and over these four games to what I saw in Karlsson’s first season as a Shark, I’ve clipped just one game from that season: A rather bland 4–3 loss to the Winnipeg Jets where Karlsson recorded one assist and was a -1.

Erik Karlsson in 2018

I chose this game and only this game because despite how forgettable it may appear on the surface, I recall from memory alone just how dominant Karlsson was. Not only did he aggressively challenge the flow of play in the manner that he still does today, but he bent the flow of play to his will through succeeding on such a high percentage of these challenges and managing the puck well after doing so. Because of this, the Sharks consistently out-shot their opponents by a comfortable margin when Karlsson was on the ice, a theme present throughout his prime. (More on this later).

Karlsson was never a perfect player, and never looked entirely comfortable when opponents attacked him with speed, but he managed much better then than he does now. This is highlighted at 0:16 when he stands up a trailing Josh Morrissey, preventing a shot and then sprinting out of his zone to lead a rush in the other direction. In addition, he frequently recovered dump-in and chip-and-chase attempts and skated himself out of trouble before opposing forwards were able to establish a forecheck. I won’t break down every play in this video, as I believe simply watching it after watching film from this season makes it clear that he’s just not playing at the same level.

While I’m very confident that these weaknesses in Karlsson’s game do exist, and that the appearance or exacerbation of them are the direct cause of the massive decline in his macro-level results, I’m a bit less confident in my root cause analysis of what has created these weaknesses in the first place. With that being said, here is what I believe has caused those weaknesses themselves:

• I believe Karlsson is succeeding on a significantly lesser percentage of his challenges because his skating no longer allows him to execute them at the exact moment he must.
• I believe Karlsson is doing a significantly worse job of managing the puck because he is no longer able to skate his way out of danger as efficiently, which also means he can no longer keep opposing forwards and defenders as honest as he could when they knew they had to occupy his skating lanes and not just his passing lanes.
• I believe his inability to challenge opposing players who attack him with speed is directly related to his inability to execute on these challenges, although he also did a much better job of pivoting and keeping those attackers to the outside without challenging the flow of play than he does now.
• I’m not sure what has caused him to take inefficient skating paths; perhaps it was that he always took them and his skating ability just allowed him to compensate for them enough that I didn’t notice them in the past. Or perhaps his instincts have declined as well.

While my analysis alone makes it clear enough that these weaknesses exist, a skills coach may be able to provide a better root cause analysis of these weaknesses than I can, and perhaps even make a prescription to improve upon them. I’d love to hear Karlsson’s game broken down from that perspective.

Regardless of the root cause of his decline, though, it’s clear to me from watching the games alone that Erik Karlsson is nowhere near the player he was in his first season as a Shark. It’s also clear to me from nothing but the eye test that there’s no path to positive on-ice results for a player who directly contributes to this many dangerous scoring chances against. Even if Karlsson fully matches the offensive impact he made in his prime — something he hasn’t done as of late, and something which is exceedingly unlikely given that he can no longer bend the flow of play as he previously could — I still believe his net impact on the game would be negative at this point because his defensive impact has declined to such a massive degree.

Crunching the Numbers

My opinion is that four games into the season, when the goal is to evaluate an individual skater, the eye test reigns supreme over just about any statistic. The way that I see it, there are just enough contextual factors at play to completely throw off any single metric, but there aren’t nearly enough contextual factors to properly account for all of them.

Having said that, I find it very difficult to use the eye test alone and state that any point has been proven. Maybe it’s just that I’m a stats guy by nature and by trade, and that I’m used to taking positions where the meat of my supporting arguments are data-driven, but I don’t feel comfortable presenting such an assertive thesis using film alone and then calling it a day. After all, every player makes some good plays and some bad ones, and it’s theoretically possible that a player could make as many bad plays as Karlsson does and still have a net positive impact on the game through the good plays they make.

I also think it’s important that I do use the data at hand to check whether my positions are not being skewed by any biases or pre-conceived notions. While the eye test will ultimately take precedent in this particular use case, I’d like to ensure that it either lines up with the data, or that I have a good understanding of why it doesn’t, before I assert my thesis with such confidence.

Give it to me raw

Before getting into any sort of deep statistical analysis, I’d like to keep things as simple as possible and simply look at some plain, raw metrics that describe how well the San Jose Sharks have controlled the flow of play at 5-on-5 with and without Erik Karlsson:

With Karlsson on the ice at 5-on-5, the Sharks have attempted 46 shots and allowed 74 (38.33% CF). They have generated 2.34 expected goals and allowed 2.96 (44.12% xGF).

With Karlsson on the bench at 5-on-5, the Sharks have attempted 100 shots and allowed 117 (46.08% CF). They’ve generated 5.07 expected goals and allowed 4.96 (50.52% xGF).

On the surface, these numbers are a major indictment of Karlsson. The Sharks do a poor job of controlling play with him on the bench, and then fall off a cliff once he steps on the ice. Making a bad team worse has always been a kiss of death.

It does bear mentioning that are a few pieces of key context missing here. One is that the Sharks have held the lead for a much larger percentage of their ice time than they’ve trailed. Another is that they’ve been on the road for three of these games against exclusively rested opponents, playing a back-to-back in one of them. One piece of context that is specifically relevant to Karlsson and how he looks relative to his teammates is that he has mostly played alongside Jacob Middleton, a 25 year old with 18 NHL games to his name. Karlsson and the Sharks are likely not quite as bad as the numbers suggest.

The sample here is also small enough that the numbers could just be a fluke. It could be that through some combination of random chance and unflattering context, Karlsson’s individual impact has actually been net positive over these four games. However, the raw metrics alone look so horrible that we can be rather confident this isn’t the case; it’s just not likely that a player with a positive impact would see their team perform poorly with them on the bench, and then perform so much worse once they step on the ice.

Karlsson’s on/off-ice splits are a stark contrast from what they were at his peak. Although prime Karlsson’s impact on shot quality may not always have been elite, his impact on shot quantity always was. At 5-on-5, his CF% through every single season as a Senator and his first two as a Shark was always at least 2% higher than his team’s CF% him on the bench, and generally around 5% higher on average. I believe the primary reason for Karlsson’s control of shot quantity was that he bent the flow of play in his team’s favor, but his relative lack of control over shot quality was because he did so through risky challenges which occasionally led to odd-man rushes against when they failed. The result was that the Senators and Sharks (in his first two years with them) did not always do a relatively great job of out-scoring their opponents, or out-chancing them after adjusting scoring chances for quality.

However, Karlsson’s inability to consistently drive shot quality and goals to a major degree is merely a minor quip when compared to the fact that he has no longer been able to drive shot quantity either. In both the 2020–2021 season and in this one, his teams have had a higher CF% without him. I believe this is primarily because those risky challenges are failing more frequently when they happen, and also because they are happening a bit less frequently.

If these numbers do not improve over the course of 82 games, though, then we can be certain Erik Karlsson’s impact has been bad. There is no good NHL player whose team controls 38% of the shot attempts when they’re on the ice and 46% of the shot attempts when they’re on the bench. The players for whom that is their true level, if any even still exist, are some of the very worst players in the league, and any robust model worth its salt will support that conclusion.

Wins Above Replacement

Models like my Wins Above Replacement model use regression analysis to account for some of the aforementioned contextual factors that are not considered by raw on-ice metrics, such as teammates, opponents, and deployment. All else being equal, they are usually a large improvement on such rudimentary analysis as the on/off-ice splits, and allow us to state our conclusions with more confidence.

Unfortunately, building and releasing the results of regression models such as Wins Above Replacement when the entire league has played fewer than 100 games is worse than an exercise in futility; it’s an irresponsible act of charlatanism. I believe the outputs of such models are snake oil. Therefore, I don’t have Wins Above Replacement for this season alone, and I won’t until around January. This means we can’t use Wins Above Replacement to accurately estimate Karlsson’s contributions in this season alone.

I won’t go into great mathematical detail as to why this is, but the essence of it is that ridge regression models are simply not stable within such small samples. It’s not that we shouldn’t draw conclusions from what happens in these small samples; it’s that these models can’t accurately tell us what happened in these small samples. If you’re interested in learning more about this, please read this Twitter thread I authored back in January:

With that being said, I do believe there is a way that one can responsibly use such a model to analyze what has happened in a very small sample, such as all games played this season (through October 22, the time of this writing): Combine the data from this season and last season, treat them as one large sample and run a mathematically stable regression model on that sample. Once that is complete, compare the results to those which use solely the prior season.

Doing so doesn’t change much for Erik Karlsson:

It’s valid to ask how much stock should be put into these numbers. Personally, I still don’t think the answer is a very large amount, as these could be thrown off just as much by new information about the players Karlsson has played with and against as it could be thrown off by his own play. It’s better to view the “Difference” in this chart as an answer to the question “how has all of this new information changed our perception?” than the question “how much has this player’s individual play improved?”

With that in mind, it is still true that if a player has shown a true resurgence early in the season, the data should bear that out to some degree. For example, four games is enough to suggest that a healthy and re-invigorated Victor Hedman has already given us a very strong reason to believe that last year’s injury-plagued season was an anomaly, and that dominance will continue to be the norm for him going forward.

In the case of the San Jose Sharks, a sizable portion of credit for the early resurgence at the team level is distributed to forwards Timo Meier and Logan Couture, who have visibly looked much better as well:

So, while differences in WAR percentile are certainly not the be-all end-all of descriptively evaluating how a player has performed four games into the season, and are prone to large error, there is a reasonable expectation of improvement for a player who has undergone the sort of massive resurgence that Karlsson purportedly has. Karlsson has not met that expectation in the slightest.

While it is difficult to express conclusions regarding four games of data with a high degree of confidence, a robust and responsible examination of this year’s data does not show any of what we might expect to see from a player has undergone such a resurgence. I believe that the rapidly deteriorating fundamental conditions of Karlsson’s game are directly responsible for the poor grades he receives across the board from the data, and unless he makes massive improvements to them, I believe he will continue to grade out poorly over any sample and any robust metric which is appropriate for that sample.

Why do People Think Karlsson Has Been Good?

Chances are that if you’ve read this far, you respect the thesis of this article. At the very least, you understand that there are a few fundamental red flags present in Karlsson’s game even if you don’t entirely agree that it’s a percentage-driven mirage. I can’t imagine somebody being unwilling to make even that concession, but willing to read this entire article.

So, if you’re mostly on the same page as me, then you’re probably asking the following question: Why do people think Erik Karlsson has been good? The answer lies in a few numbers I didn’t bring up when I crunched the data: Karlsson has scored six points in four games, and the Sharks have out-scored opponents 4–1 with him on the ice at 5-on-5.

PDO

The reason I did not bring these numbers up is because they are primarily driven by on-ice shooting percentage and on-ice save percentage: Two largely random variables which Karlsson himself (and any defenseman, as well as most forwards) have little to no control over. When these two numbers are added together, it forms a statistic known as PDO, which was made popular by Brian King, an irate Oilers fan who was frustrated that his favorite team placed high internal values on players who had high PDO and low internal values on players with low PDO, not recognizing that PDO was a largely random variable that was more in charge of the results they paid attention to — points and goal differential — than anything those players actually did. (Note that the NHL refers to PDO as SPSV%.)

Erik Karlsson’s 5-on-5 PDO is currently 112.7. His 5-on-5 on-ice shooting percentage is 16.67% in a league where the average is 7.67%. His 5-on-5 on-ice save percentage 96.67% in a league where the average is 92.3%.

On the power play, where Karlsson has notched four of his six points, his on-ice shooting percentage is an insane 40%; league average is 14.7%. On the penalty kill, his goaltenders have stopped 100% of the shots they’ve faced with him on the ice; league average is 85.3%.

To give you an idea of how wacky and unsustainable these numbers are:

• Karlsson’s prior career high 5-on-5 PDO is 1.011.
• On the power play, Karlsson’s prior career high on-ice shooting percentage is 17.72%.
• On the penalty kill, Karlsson’s prior career high on-ice save percentage is technically 100% — a mark he achieved in just over seven minutes of ice time in his rookie season — but he’s never reached 90% in seasons where he has played 80 or more games.
• The last time the NHL played a full 82 game season (2018–2019), the league leader in 5-on-5 PDO among players with at least 82 games played was Jakub Vrana with a mark of 1.043. Karlsson’s PDO is about three times further from league average than the best player’s PDO was in the last full season.
• The leader in power play on-ice shooting percentage among these same players in this same season was Steven Stamkos with a mark of 22.34%.
• The leader in shorthanded save percentage among these same players in this same season was Leo Komarov with a mark of 93.79%.

Karlsson’s percentages are running laps around his career highs and around the players who led the league in the last full season. Even if these percentages were somehow both sustainable and reflective of Karlsson’s true impact, they would clearly disprove the theory that he has turned back the clock and returned to his peak level of his play, as being outshot and riding high percentages was the exact opposite of what he did at his peak.

Before I move forward with the conclusion of this article, I must clarify the biggest problem with using on-ice shooting and save percentages to evaluate skaters is not that they are unsustainable and not predictive, but rather that they are mostly out of the control of skaters and therefore not accurately descriptive of their impact on the game. In Karlsson’s case, they are more descriptive of the upgrade that San Jose has made in goal, and the improved finishing touch of their forwards thus far. Properly understanding this distinction is key as it allows one to take the more accurate of the following two viewpoints:

1. Erik Karlsson’s point totals and goal differential are probably not sustainable, but they are reflective of large strides that have been made to improve his game.
2. Erik Karlsson’s point totals and goal differential have little to nothing to do with how he has actually played.

Some folks believe the former is a rational and fair viewpoint. I believe it is completely wrong and that only the latter is accurate. I believe the film and data clearly bear this out; the 96 seconds alone from this season which I clipped show numerous scoring chances where Karlsson was entirely at the mercy of his goaltenders, who have bailed him out early on. I am happy to hear any counter-arguments, especially those which incorporate film, but until I am compelled by one I remain very confident in my position.

What Comes Next for Karlsson?

Make no mistake: The percentages that Erik Karlsson is currently riding will not be sustained over a larger sample. As sure as the sun is to rise tomorrow, Erik Karlsson’s on-ice shooting and on-ice save percentages are to drop as he plays more games. Even those pumping his tires can agree this will happen, though they will probably convince themselves that something has fundamentally changed enough to allow him to sustain a higher PDO going forward than he had in the past decade of hockey he played, even though this has never really happened for anybody.

When Karlsson’s PDO inevitably does come crashing down, one of two events will follow:

1. If and only if the fundamental aspects of Karlsson’s game have improved to such a degree that the Sharks can and do manage to sustain out-scoring their opponents with him on the ice, and Karlsson continues to score a large number of points, fans will declare this season a resurgence for Karlsson, even if his true impact on the game is no better than it was as recently as 2019–2020 (when it was still quite good).
2. If the fundamental aspects of Karlsson’s game have not improved, his point rates will decline considerably (though likely above the extreme low they reached in 2020–2021) along with his goal differential. This will lead fans to continue to say that Erik Karlsson is washed up, and a shell of his former self; the same things that many were saying back in the early goings of 2018–2019, when Karlsson was visibly dominant but could not buy a goal or a save.

Based on what I’ve seen from the film, I don’t really see a path to #1. Karlsson’s ability to pivot quickly is poor enough that he can really only defend opponents who attack him with speed if he challenges them and counters with speed of his own, and his challenges become increasingly riskier and less likely to succeed as his skating continues to decline. In addition, while every athlete should genuinely be confident in their abilities and present an aura of confidence even when it’s not entirely genuine, Karlsson’s over the top comments made at the start of this season his awful 2020–2021 campaign suggest he hasn’t acknowledged the fundamental flaws in his game or taken full responsibility for (or even acknowledged) his poor results.

As a die-hard Sharks fan, and a big fan of Erik Karlsson, I hope like hell I’m wrong. I would love nothing more than to see him turn it around and win another Norris Trophy. I just don’t see any evidence of the fundamental changes that need to occur for that to happen.