On Tracking Scoring Chances, Part III

Today we conclude our three part series on tracking scoring chances. In Part I we covered our first impressions of the process, and what we thought we got out of it. In Part II we dove into some specific examples to look at how a variety of factors led to our ultimate decisions. Today we'll give you our closing thoughts on the value of scoring chance data, what it means, and why it may be more valuable to NHL teams than to fans tracking the game manually. Without further ado...

So what are the finals thoughts, did it change how you view the data or how useful you think it is?

RP: I don’t think it really changed my beliefs on the utility of scoring chance data, I already thought it was useful data. I did used to think that if more people tracked the data, it would be more useful. Now I'm not so sure. I have notes of potentially questionable chances that I personally didn’t consider a scoring chance and I thought that having more people judge those plays will create a more robust picture of what happened. This process highlighted how subjective these things are on an individual level, but I also had a lot of experience with that when I did the soft goals analyses so it wasn’t surprising. The degree to which the subjectivity infiltrated our analysis, and the fact that there were pairs or groups of us were clearly on the same page did make me think that it's so subjective that large numbers don't necessarily cure the subjectivity.

JP: Agreed. I wouldn’t say that it changed my thoughts on the utility of scoring chance data, generally, but it hammered home that scoring chances are, ultimately, subjective - as I fret about whether a guy’s release point was inside or outside of the "home plate" area, what I decide impacts the data itself. And that, in turn, highlights my bigger takeaway (which is confirmation of what I’ve always thought about scoring chance data), namely that counting up chances is all well and good, but not all chances are created equal and treating them as such doesn’t necessarily faithfully represent what’s happening (a tap-in from the side of the net isn’t the same as a backhand flip from the dot with a defender in the way). Chances should be graded, which would be all well and good for an individual team’s analysis (which I’m sure is exactly what they do), but would be hell in trying to compare players across teams. Point being, great chances are going to be "scoring chances" on everyone’s sheet, but how many not-so-great chances get included can make big differences.

GOULD: It didn’t make me question the value of scoring chance data in general (I think it could be very valuable.) What it did was make me want to figure out who did the scoring and whether they were experienced and whether their data is generally accepted. Because I was terrible at it, and I wouldn’t want anyone relying on my data. One thing became very clear right at the beginning: with my personality type, I can either enjoy a game or score it, but I can’t do both. This felt like work. I scored the game two months after it happened, and there was no suspense about the outcome -- I had to click right on the final score to load it up on GameCenter live (something I’d like them to change so that you can, you know, watch a game after the fact without having it ruined before you start). So I wasn’t watching play or systems or paying attention to the flow of the game or anything fun like that. I was just focused on whether there was a chance or not. I found my focus wandering at times, so I didn’t do a great job, though I did rewind and rewatch lots of plays and I generally took it seriously and tried to do a good job. This is probably not something I’m going to do again. And the folks who have been doing it all the time, like Neil, have even more of my respect than they had before. In the immortal words of Dr. Klahn, you have contributed a data set of enormous magnitude, and you have our gratitude.

SME: I don’t think there’s necessarily much utility in the raw numbers for scoring chance data. However, I do think there is very good utility in the ratios recorded by the team. As already touched upon, they are very subjective, which can change the numbers quite a bit. And as JP noted, the way they are recorded is far too binary for scoring chances; it assumes that all chances are created equal. All chances are equal, but some chances are more equal than others. It’s difficult to record all of the chances in a way that makes it easy to compare in such a broad manner across teams and players, especially when taking in the subjectivity of it, and all of the moving parts involved in the game.

renstar: This project changed how I viewed the data in the sense that, beforehand, I had very little appreciation for what the scoring chance data gave us. I found that actually tracking the stat gave me a better understanding of what the data describes and found that the action of tracking the chances gave me a better feel for how the control of the game ebbs and flows (maybe just because I was watching more closely). But this project as a whole made me more wary of the individually tracked data sets. I mean that as no insult to those who did the tracking: it was definitely yeoman's work, and we learned so much from the two-season effort. Rather, there are too many places for human decision to come in to play, and too many seemingly valid definitions of a scoring chance to convince me that there is one that captures something that correlates cleanly to possession. (More on this in a moment.) In a way, I'm actually glad that it was eventually found that scoring chances track with Corswick, as the Corswick stats can be adjusted for home/away/arena biases and are based on globally available data.

Briefly going back to the idea of possession...before we found out that the scoring chanes project ended, I went back over the first period of the game that we all tracked (so this is the king of small sample sizes) and looked at two easily defined, unambiguous measures of possession: the zone that the puck was in and the team who had control of the puck. The next image shows possession in the first context. Red is when the puck is in the PIT zone and WAS 'has' possession, white is the neutral zone, gold is PIT possession (puck in WAS zone), and black lines are stoppages. The scoring chances determined by this project are plotted for each tracker.

Similarly, I looked at actual possession, as in which team had the puck on their stick. Again, red is WAS possession and gold is PIT possession with the scoring changes plotted over it.

As I said, these are small sample sizes so I'm not drawing any strong conclusions, but it is not completely clear to me how scoring chances correlate with or drive possession.

D’oh: This exercise changed my thoughts on the utility of tracking chances, and I’m sad to say it was for the worse. I don’t think it’s an entirely fruitless endeavor, but I’ll echo what JP said above - it’s far too subjective and not all chances are created equal. I found myself marking a lot of chances as "marginal" to reflect that, while I thought it was an above-average chance, it wasn’t almost a goal. Moreover, a more liberal interpretation of a scoring chance would seem to heavily favor a puck-possession team or a team that takes many shots, whereas a more restrictive interpretation might favor a team that worked to create excellent chances. This means that comparisons between teams (and between players on different teams) are largely useless unless the same person is tracking both teams - even then the validity of the data would depend on the styles of the teams being tracked and the tendencies (liberal or restrictive) of the person doing the tracking. I’d wager that teams have their own internal tracking system, with grading scales and multiple people to track each game.

GOULD: There are ways to deal with the subjectivity. First I think is experience. I bet folks get more consistent with one another at scoring games just by scoring lots of them, even if they don’t compare data or anything. Second would be some kind of cross-check that doesn’t currently exist. I’m actually a published author on a couple of statistics-based legal articles: we coded hundreds of trade secrets cases for certain objective factors and then presented an analysis (email me if you want to know more). One of the things we did was cross-check a few of each other’s cases and talk through any differences of opinion on how the case should have been coded. It made us more consistent with one another. If scoring chance trackers would organize and coordinate, this would actually be pretty easy to implement. It would make the data better, but I don’t think it’s absolutely necessary. For now, we should treat the data like "hits" and even "shots" -- data where there’s a certain range of subjectivity so you have to take it with a grain of salt, but ultimately the world is better for having the data available and you just have to do your best to make sense of it in a fair and accurate way.

JP: My second big takeaway here flows from the first one, and it’s another point that you touched on, Rob - given the subjectivity involved, the more people tracking this stuff, the better the data will be. Even just having a second or third scorer will make the resulting numbers more useful. As fans, it’s not a futile exercise - it’s important crowd-sourced information that can help provide a better understanding of what we’re all seeing.

SME: The ratios and numbers involved strike me as lending themselves to being overemphasized in the same manner that faceoffs are, though scoring chances are much more important. But when the ratios have a player getting 45% of the chances when he’s on the ice, that seems bad, until one looks at the fact that with a couple more chances, they’d be at 50%. It strikes me at being useful at comparing players on the same team, though not necessarily across teams. Like so many of the stats that already exist.

D’oh: One thing that struck me is that the offensive data for defensemen is probably skewed by the fact that a lot of good chances are off the rush, and the defenseman (men) who started the play with outlet passes have left the ice on a change by the time the chance is registered. I counted a few instances of this phenomenon during this game and, as SME points out above, that can shift a player’s chance ratio from positive to negative pretty quickly. I’ll also point out that I found one pretty large error in the RTSS data that timeonice.com uses to track who’s on the ice for chances - given the small difference between a player being a "plus" and being a "minus" I think this makes me more wary of using this data for analysis of individual games. Over the course of a season I imagine those errors wash out. Another take-away is that more and better camera angles would really help. There was a chance at 5:31 of the 2nd period where Steve Sullivan and Richard Park managed to create a little 2-on-1 in the slot against John Carlson. From the original angle (shown below), I logged it as a chance, albeit a marginal one.

Later, the broadcast showed a view from behind the net. From this second angle (shown below), it appears as though Carlson has the passing lane almost totally cut off and Sullivan has his head down. It would take a great pass to get to Park, and because Sullivan would have to put the pass somewhat behind Park (in order to clear Carlson), Neuvirth might have enough time to get over to make the save. I still think it’s a scoring chance, but the second angle really gives much more information.

Rob: More camera angles would definitely help, I just don't think this is the best example to make your case. A two-on-one in the slot is going to be a scoring chance. Period. You can applaud Carlson for playing the pass well (as is his job) but the fact remains that the Penguins should have had a high quality shot on net; the Caps were saved by an extremely poor decision from Steve Sullivan more than anything else. But I digress.

There's a bit of a paradox in scoring chance evaluation. On the one hand, you want more eyes to build a more complete picture and to wash out individual biases, but on the other hand you can only truly get a consistent definition to apply if you use a single person to define a scoring chance. I think this is where the main difference between fan tracking and team tracking lies. Teams only need the definition they are comfortable with, and their scouts can calculate the data as they are comfortable; the same set of eyes is creating the entire data set. With fan tracking, you have a different set of eyes tracking each team, and they are simply not going to be able to apply the same definition. If they do manage to agree upon a mechanical definition of "scoring chance," they'll lose some specificity and undercut the value of their own data. As Dan Bylsma noted: "There is more to it than just location," said Bylsma. "Way more. It's not just odd man rushes. We have a certain area [on the ice] we consider to be a scoring chance. There are circumstances that if it's outside that area it could still be a scoring chance, like a wrap-around. Some wrap-arounds are chances, some are not."

In order to apply that highly specific, context-dependent definition of a scoring chance, you need a dictator to make the final ruling, even a tightly knit coaching staff is going to disagree, and several eyes can muddy the picture just as easily as they can clarify it. In an NHL organization, one person has the final word, but that's not the case in fan tracking. For an NHL team, it's the head coach that has the final word. Bylsma continued: "There are some times as a coaching staff, maybe twice a game, we have to get together and decide if there was a scoring chance or not. The reason it is or isn't is not because we all agree, it's because I say it, because I'm the last guy to say it is or not." There's no way to make those close calls in fan tracking, so it ends up aggregating individual biases without identifying or controlling for those biases.

On Tracking Scoring Chances, Part II

Yesterday we talked about the process of tracking scoring chances, including the difficulties that arose and the insights we gathered. We showed how a different set of eyes could yield different results, and how a particular team or player might be viewed differently based on those results. Today we'll get into some of the specific chances that we disagreed over. The following examples are grouped based on postural similarities. Location is always going to be key when evaluating a scoring chance, so we'll look at how some other factors influenced our grading of these chances. The factors that will be focused on below are puck possession, puck movement, and defensive posture (of the defense and the goalie). We'll provide video of some of the chances that we disagreed on internally, and then provide short statements defending our ultimate decision. Let's start with a chance that was referenced in Part I, the Mike Knuble backhand shot attempt from the slot. First the video:

Join us after the jump for our thoughts...

SME, Renstar, and GOULD marked this as a chance.

JP: A wild backhand through a maze of sticks and players? It’d have been a miracle goal and not the result of converting a dangerous chance.

Gouldie: This was a tough one for me. Richard Park is all over Knuble like pink on an AHL Valentines Day commemorative jersey. Knuble can’t get a good shot off. Ultimately, I think it was Jeff Halpern’s derriere in the face of Marc-Andre Fleury that swung this to "chance" for me. I’ve almost never seen a player as mobbed as Knuble was score a goal, but Halpern’s screen and the chance for a tip-in make that a dangerous play.

D’oh: Echoing J.P., a goal from this shot would have been more a matter of luck than of skill. Maybe Halpern deflects that shot in, but this was a "just get it on net and hope something good happens" type of play.

SME: I’m with Gould on this one. If Fleury has a clear and unobstructed view on this, I wouldn’t consider this a chance. But Halpern is sitting right in front of Fleury creating a good screen for Knuble. Similar to the Letang goal, the traffic in front of the net makes a shot more dangerous than it otherwise might be. The high chance for a tip or a rebound, especially since the PIT D that was on Halpern left him to alone in front of Fleury. Considering that it was a backhand through pretty much every player on the ice, it would have definitely been quite a bit of luck had it gone in. That said, I still counted it as a chance as he got the puck through to the net, right from the slot, he didn’t flub the shot and made Fleury make the save. That kind of play I wouldn’t be surprised if the puck had gone in.

Rob: I just don’t see this as the kind of play where the Capitals broke down the Penguins D, or the Penguins made a defensive mistake. Knuble is able to walk the puck out of the corner, but he never has a clear look and this isn’t the kind of play where I’d expect a goal to result. The PIT defenseman was with Knuble the whole way, and even had a little help. I could see Bylsma not being very happy that Knuble drove the puck to the slot, but it’s not a particularly difficult backhand for the goalie to handle and the D is all over Knuble (who isn’t known for his shot, anyway). This isn’t the kind of play that really pulls me off the edge of my seat. I think D’oh makes a good point about it being a "hope" shot: is it really that much more threatening than a point shot?

renstar: For me, it didn’t have anything to do with who the shooter was or that Fleury had an obstructed view. The shot wasn’t the most important thing for me on this play. The chance for a deflection is what stood out. Without Halpern there for the chance at a deflection, I probably don’t count that. This is a good example of what I meant above by a reasonable chance to play the puck in the danger zone.

Let's take a look at a two more potential chances from right in the heart of the slot. First up is a play from the first period involving Jason Chimera and Alexander Semin:

SME, Renstar, and GOULD marked this as a chance.

renstar: This one should be among the most controversial chances we consider. It was also the first one I encountered, so I found myself using the short rewind feature of Gamecenter to watch it over and over, trying to figure out how I felt about it. I ended up going with a chance here simply because Semin was a quarter step off from getting there, and that short step is, in my opinion, no different than having the stick turned slightly wrong and stuffing one into the goalie’s pads. The Penguins won’t want to keep giving that play up, because if it connects, it is a goal.

JP: It should have been a chance. A better pass and it is. But no Cap ever had the puck in a good scoring area, so it’s not in my book. At no point was a shot even attempted, so I can’t call it a scoring chance. But does that mean that an otherwise perfectly executed 2-on-1 where the puck skips over the would-be shooter’s stick isn’t a chance? I’d argue that it isn’t, but that the result is not capturing what’s really going on on the ice in terms of quality offensive opportunities.

Gould: If a guy has an open shot and whiffs the opportunity, that’s a blown chance. If a guy can set up his teammate with a pass and the timing is slightly off, I think that’s got to be a chance too. I don’t think any kind of pass/shot distinction is helpful. Chimera and Semin beat the defenders and had an opportunity for an easy goal if the timing had been right. Semin’s nearly alone with the goalie out of position and the puck on his stick. Seems like a chance to score a goal to me.

D’oh: Football has the "catchable ball" clause for pass interference, and I’m invoking the "receivable pass" clause to strike this down as a scoring chance. Semin can’t get his stick on that pass, and even if he does, he’d just be poking at it. Fleury has plenty of time to get over and square up to the puck. Therefore, it’s not a scoring chance in my estimation.

SME: I’m with Gouldie. It was still a chance despite the blown pass/reception. We see goals all around the NHL on that sort of play all the time. Even though Semin isn’t able to get his stick on the puck it’s still a very dangerous play.

Rob: I’m with and against J.P. I’m with in the sense that close isn’t enough. Almost isn’t a chance. But simply because it’s no shot doesn’t mean it’s not a chance. If bad ice made the puck bounce over Semin’s stick on an otherwise tape-to-tape pass then I’d probably call it a chance. But it wasn’t a threatening pass, and you could argue that if the angle of the pass changed such that the puck could have gotten to Semin, maybe the D intercepts. No chance.

Compare the above play to this next play. In contrast to the play above, this following play will involve clear possession in the slot. However, contrary to the Knuble chance above, there will be no shot on net.

Rob, Renstar, Gould, and D'ohboy counted this as a chance.

renstar: I had this as chance because there should have been a goal scored. It was an odd-man break and the puck was put on net. A better shot or one more pass and it is a goal. Not the play you can give up if you want to lead hockey games.

JP: I treated this the same way I did the Chimera-Semin 2-on-1 - no shot, no chance, even though the opportunity for both was right there. Tough call, I guess.

Gould: Pens broke down the Caps’ defense and were one easy pass away from an easy shot into an open net. This was a golden opportunity to score a goal.

D’oh: At first glance, this is a no-brainer scoring chance, but watching the replay it’s not as clear-cut. Yes, it’s a two-on-one, but it’s very brief - it’s not as though Sullivan and Park had a chance to really set up. This is probably why Sullivan and Park are skating parallel to each other, which allowed Carlson to simultaneously cut off the passing lane and pressure Sullivan. It’s a scoring chance, but it’s not a great one.

SME: I didn’t have it counted as a chance, but treating it the same way as the Chimera-Semin chance, I have to revise my decision. It’s something that should, or very easily could have been a goal with a quick cross-ice passing play on a 2-on-1. The passes and shots could have certainly been better, but that was still a dangerous play that often leads to a goal.

Rob: This is why I disagree with J.P. on the Chimera/Semin play. There was no shot, but a clear two-on-one in the slot is always a chance to me, no matter how you look at it. The offense earned an opportunity, the defense broke down, the defensive coach will hate it. Sullivan made a bad play, but an objectively dangerous situation was created for the Pens. The replay with the high angle behind the net really confirms just how much real estate the Pens had with the Caps vulnerable.

Let's transition and look at another factor in judging a scoring chance: defensive posture. First up is a shot from the left wing circle by James Neal:

J.P., SME, and Gould counted this as a chance.

JP: This was similar to that Dupuis chance we discussed earlier [ed. we had that play counted as a scoring chance 5-1, SME in dissent] - it’s a scorer in a decent spot with a quick shot. Is it no chance because he flubbed the shot? That seems to be ex post facto - the chance, in my opinion is when he has the puck in position, whether he buries it, shanks it or otherwise.

Gouldie: Nice passing leads to an open shot on net. Sure, it was a bit far out, but I’ve seen that goal scored in the past.

D’oh: This is similar to, but significantly different from the Dupuis shot that I earlier rated as a chance. In both cases, the puck comes from behind the net to a Pens player around the circle for a quick shot. In the Dupuis chance, the puck comes quickly from almost directly behind the net. On this play, the puck comes out to Neal a bit slower and from the corner. The former causes a more rapid change of angle for the goaltender and defensemen. The biggest difference, however, is the presence of defenders. The Caps have Neal pretty well bracketed here, with Knuble taking away his stick and Laich moving in to block his shot. This was much better defensive coverage than that provided by Wideman on the Dupuis chance, and it’s what prevented Neal from getting away a good shot.

renstar: Any power on this one and it is a chance, but it just wasn’t there.

SME: When I re-watched this, I had the same reaction as J.P. But watching this in tune with the game I considered it a chance because of the shooter and location. Pens work the Caps down low to get him open in the slot area for a one timer that he misses, but is still a chance.

Rob: This is one of the rare cases where I differ from J.P. Yeah, it’s close, and similar location to the Dupuis chance, but the D is on Neal and that’s a huge part of the reason it’s a flubbed shot. This isn’t a pure flub like the Michalek shot. While Neal is the kind of guy that coaches say "don’t let him beat you," the D being on him is important to me, so I have it as no chance. Or what D’oh said.

Unlike the Neal shot, this next play features an open shooter, and a different overall defensive posture:

Gould, Renstar, and SME counted this as a chance.

renstar: I counted this due to the one-timer, plain and simple. That sort of puck movement is going to result in a goal eventually.

JP: I might have counted that as a chance if it was a Shea Weber teeing it up, but that’s still a fairly unscreened shot from way downtown - a goalie with a working groin shouldn’t have much trouble with it, despite the movement.

Gouldie: The two Pens down low are closer to Neuvirth than the two Caps defenders -- not something you want to see. Mchalek flubbed the shot. If he doesn’t flub, but instead takes a hard, low shot, and there’s any kind of mistake by Neuvirth on the rebound, then that’s an easy goal for one of the two guys in front of the net.

D’oh: Sullivan’s pass across to Michalek wasn’t great, and Michalek couldn’t really get good wood on his shot. Kunitz is in front of the net, but Neuvirth sees the shot the whole way - not a scoring chance.

SME: I counted this as a chance because it was a cross-ice one-timer, even if it was a poorly executed one. It wasn’t a very good chance, but I still counted it as one. (Also, I’m fairly certain that it’s Niskanen, not Michalek...)

Rob: I’m with J.P., the fact that it’s not a very difficult shot (or shooter) is important to me on this one. In terms of culpability, it’s not great D for the Caps, but Michalek isn’t the kind of guy coaches say "don’t let him beat us" so I’m OK with this.

Finally, consider this last opportunity and how the goalie's posture impacted our thought process:

J.P., Rob, and Gould counted this as a chance.

renstar: I had this as a potential chance but ended up not counting it because the angle was too sharp, Sullivan was too close, and there just wasn’t any dynamism in the play. If he cut inside and not out, it probably would have counted.

JP: What was Hamrlik doing there? Jesus. Anyway, it’s a chance in my book, even if the angle was a bit severe - there’s some space upstairs and maybe some five-hole; if the shooter is unscreened from there and there’s net to shoot out, it’s a chance, says I.

Gouldie: This is like the Halpern play. The forward beats the defenseman and ends up one-on-one with the goalie. Seems like that’s always a chance to me, even if the forward is a little off balance and the angle is less than ideal.

D’oh: The angle was just too sharp on this shot. Maybe Sullivan would have had a better chance if he had a teammate right on Neuvirth’s crease, serving as a distraction. Even though Hamrlik looks like a turnstile, he at least prevents Sullivan from cutting back to the slot. Neuvirth could hug the post and Sullivan, as a right-handed shot, has pretty much nowhere to go but top right corner under the bar. That’s an unbelievably difficult shot and therefore not a scoring chance.

SME: Sullivan made the Caps D look silly there, but I never considered it a chance because he was at a very tough angle when he finally got open enough to shoot and Neuvirth was with him even if there were a couple gaps on his person. That’s not a goal I would ever be OK with a goalie giving up, therefore I don’t give it the distinction as a scoring chance.

Rob: Sullivan is inside the home plate with no D on him. Neuvirth is square, but it’s a shot that you see end up in the net not infrequently. It isn't quite as dangerous as Dupuis' sharp-angle chance [ed. scored as a chance 4-2] so I can see how it’s borderline; this is the kind of chance that makes me really wish we could implement a system that was more than binary. This is a weak scoring chance, in my book.

Now, having seen some concrete examples, you can see the thought process behind our determinations. There are so many factors at play (and we only really touched on a couple of them) that it was tough for us to develop a clear-cut definition of a scoring chance. Obviously location is going to be a big part of it, but when you start considering the context of the defensive and offensive players, the location-based definitions begin to breakdown (at least in our opinion). Tomorrow, in Part III, we'll wrap things up with our concluding thoughts on the process and the value of the information.

On Tracking Scoring Chances, Part I

Those that have been following the ongoing development of advanced statistical analysis in hockey are aware that a group of bloggers got together and began tracking scoring chances during games over the past two seasons. The project offered promise, including some insight into the ever-popular debate regarding the existence of shot quality. After the second season of tracking scoring chances, the group decided to shut the project down, as scoring chances seemed to tell us no more than the pure Corsi statistic was telling us.

While the statistically inclined fan community may have determined that there is no value added (for their purposes) by tracking scoring chances, NHL front offices have made and continue to make use of them. Recently, Ted Leonsis revealed that the Washington Capitals' coaching staff tracks scoring chances to evaluate players. Peter Laviolette has (possibly inadvertently) admitted that the Philadelphia Flyers also track scoring chances to evaluate his team's play. Among a list of teams that openly use advanced statistical analysis, Adam Gretz has explicitly identified the Pittsburgh Penguins and Phoenix Coyotes as teams that use scoring chance data. Given how secretive teams are about what stats they use, it's very likely that at least a handful of other NHL teams are tracking scoring chances.

So where is the disconnect? Why do teams find value in the scoring chance data, while the scoring chance project determined that scoring chances don't tell us anything new? To get some insight into the process, J.P. and I have recruited some regulars from our comments section to do our own case study on scoring chance data. Those who venture into our comments will already recognize these contributors, but for everyone else let me welcome (and thank them for their help) Gould Old Days, Steckle Me Elmo, renstar, and D'ohboy. The six of us each independently charted scoring chances for one Capitals game, the game against the Pittsburgh Penguins on January 22, 2012. Join us after the jump for our initial thoughts on tracking scoring chances and the process in general.

Q1. Before we reveal to each other our specific observations on the game, what are your general thoughts after tracking scoring chances for the first time? Easier or harder than you'd thought? Did it do anything to inform or impact your decision on the utility of the scoring chance data you see out there? Any and all thoughts welcome...

RP: I think it’s harder than I expected. You can’t zone out or get caught up in the action too much because you always have to have your mind focused on tracking the scoring chances (which are generally the most exciting part of the game and thus easiest to get distracted from your task). I think it’s probably harder to track live (which I did) than on delay because when it’s live you still have the stress of not knowing the outcome of the game. On delay you’ve already seen the interesting parts, know the result, etc. so maybe it’s easier to focus on identifying scoring chances.

JP: I’m with you - it was harder than I expected, and that definitely relates to focus (and it made me realize how much I zone out at times watching the game unburdened by such responsibilities). I didn’t track the chances live (I went back and watched it on DVR a few days later; and I agree that not worrying about the outcome likely made it easier to focus on the chances themselves) and made liberal use of my "back a few seconds" button - there were plenty of shots from right around the perimeter of the scoring area that I wanted to re-watch. Granted, I likely obsessed over these more than I should have (and I’m sure I’d get better at the whole exercise with practice), but I wanted to try to be as sure as possible.

SME: It was pretty much what I expected. The only difference was having to stay more focused on the game and having a more one track mind for what you’re looking for. Two caveats come with it though. I paid less attention when the play was in the neutral zone, so I may not have seen the plays develop as much. I also watched it on a GCL replay, so I wasn’t worrying about the outcome, and, like JP, was afforded use of the back button.

renstar: I found it to be hard at first, but once I got into a zone it got much easier. In many ways, it felt the same as refereeing a game; I was completely dispassionate, yet I had a stronger feel for the flow of the game. In the first period, looking at my paper and seeing nothing but "P" in the first column hammered home how little control of the game the Caps had at that point. I also consciously watched the game with the sound off, as I didn’t want crowd or announcer noise to bias me either way. After tracking chances on delay with GCL, I don’t think I could track chances live. I rewound the stream far too many times to verify what I thought I was seeing. It took a lot of effort to be consistent, though any difficulty I had trying to be consistent about all of this probably goes away with more experience.

D’oh: At first, the coding of chances in itself seemed to be pretty easy. I went in with a clear image in my head of what I thought a scoring chance was, along with the factors that I planned to apply in the grey areas (Was there traffic? Was the traffic effective? Was the pass a good one? Was the shot on a backhand? Was the shot contested? Etc.). Still, there were some grey areas that I couldn’t resolve. For instance, was a scoring chance dependent on the players involved or was it purely based on the physics of the matter? That is, would a 45-foot wrister through traffic count the same if it came off Jeff Halpern’s stick or Alex Ovechkin’s? Ultimately, I went the route of common sense and made my chances dependent on the shooter. What really shocked me was the degree of focus it took to watch the game in this way. My mind started to wander in the second, and then it took a couple five-minute breaks in the third. I found myself rewinding and re-watching a lot of sequences. Even with all the rewinding, there was a chance that I forgot to log (which I later corrected). With practice and more focus, I think I could at least become consistent at logging chances, even if others might dispute my accuracy.

GOULD: I came into this with some vague sense that it would be pretty easy and straightforward -- something you can do while watching a game you would otherwise be watching anyway. That was wrong. Everyone keeps using the same word -- focus -- and there’s a good reason for that. I’m used to tuning out hockey, maybe writing a comment on a gameday open thread or getting up to go do something else for a bit. You can’t do any of that and you can’t zone out when you’re scoring chances.

Q2. What was your general focus when you were trying to determine scoring chances? Was it a strict geographical focus, anything inside the "home plate" area was automatically a scoring chance? What were factors that informed your decision to discount home plate shots or to count a shot from further out?

RP: My focus was much more on dangerous shots (as Knee High has called it, "the pucker test"), or shots against that tend to make coaches angry. I did not automatically code every shot inside the home plate area as a scoring chance. One mitigating factor that would switch a home plate shot to a non-scoring chance is if the D was in position to block the shot and did block the shot. To me that indicates good defensive coverage and isn’t necessarily a breakdown that left the offense open to exploit. I can think of at least one specific example of that from this game. I don’t think I was counting any scoring chances from outside the home plate area. To be a scoring chance from that far out it would probably have to include both a cross-ice passing element and a hefty screen.

I also tried to evaluate things ex ante, not ex post. So you ask whether you would be comfortable giving up that shot again. I don’t have Letang’s goal as a scoring chance, for example. It was a good shot, there was net presence, but Neuvirth saw the puck and just guessed wrong. The fact that it was a snipe doesn’t change things, that’s just good execution and a good break for the Pens, but not the kind of shot I’d consider a true scoring chance.

SME: I did it pretty similarly, using "the pucker test" but I was a little more lenient. I counted Letang’s goal as a scoring chance; he got a good shot off from the middle of the ice, with traffic, and got it through. If it had gotten blocked or missed, I wouldn’t have counted it, but the fact that he made Neuvirth have to make a save (that he didn’t make) under tough circumstances made me code it as a chance (however, that is far from the only criteria). When coding the chances, I took into account goalie position, shooter, shot circumstance, and defensive positioning. In general, I looked at it as if I were the coach - would I be happy that my team just gave that shot (in most cases) up? If the answer is no, and I think my team could, and should, have played it differently, then I coded it as a chance, for the most part.

JP: I tried to do it more by-the-book - if it was in the "home plate" area, it was a chance; outside, no chance. There were certainly times I was thinking, "Well, the goalies in position and there’s a defender between the shooter and the netminder and backside pressure on the shooter... but it’s a "scoring chance." Which, I suppose, is another general point to consider - perhaps not all scoring chances really are. Conversely, there may have been some from outside the "home plate" area that, due to the pass that set it up, the goalie’s position, and the shooter, might not have technically been a scoring chance... but certainly were. Yeah, I’m thinking of Alex Ovechkin’s goal - I had that outside the scoring chance area (barely), but it was a chance (and a goal). (And I agree on the Letang goal - I didn’t have it as a chance.)

SME: Another thing I looked for was a deliberate play, or attempt to make a play towards the net. If there was a pass to a player with an open net, and he mishandled or misshot the pass, I considered it a chance, depending on how close it was, of course. But if the puck took a flukey bounce towards the net in a manner clearly not intended by the player, it wasn’t a chance, regardless of how close it was. There had to be a certain level of intent to get it to the net.

GOULD: My standard was "if what that guy was trying had worked, that would have been a goal" or, in other words, "I’ve seen goals scored that way before, and it wasn’t a fluke." I didn’t focus on home plate or anything like that. If it felt like an above average shot or an above average pass would get through, then I called it a chance. And yes, I said "pass." A lot of what I called chances were plays where I thought two offensive players had isolated one defensive player, and they were one pass from having someone alone in front of the goalie with the puck on his stick. Like when there’s a guy behind the net, a guy in front of the net, and one defenseman between them. I think those bang-bang plays are among the most dangerous in hockey, and the fact that the puck didn’t get through doesn’t mean it couldn’t have. Obviously, anyone who looks at the stats will see that the result was that I was much more generous with chances than everyone else. Take this play: The Caps generate an offensive-zone turnover, and Brooks Laich controls the puck on the boards. He saucers a pass over to Alex Semin on the doorstep. A split-second difference in timing, and that might be a goal. Semin wasn’t ready for the pass, but if he had been he just needed to flick that one into the net. I think I scored a lot of passing plays like this one as chances. Because the way I figure, everyone else is scoring as chances events where with a better shot, there would have been a goal. But why not score plays where with a better pass there would have been an easy goal?

renstar: So my approach was to use the textbook definition as a starting point and count other situations that I thought could have reasonably resulted in a goal. I (very loosely) defined ‘reasonable’ to mean, even if there was no shot, "If a player on the offensive (defensive) team had done one thing differently for the better (worse), it could have resulted in a goal." I also counted any goal that was a result of a direct offensive attempt by an offensive player. That is, a pass to the front of the net resulting in an accidental deflection off of a skate would have counted. Weird bounces from behind the net generally would not. If the shot was from outside of the "home plate," I counted it. After all, a scoring chance is a chance to score, and if a score resulted from an offensive play, there was by definition a chance that it would happen. (Though I would probably not count a shot from the red line...subjectivity!) I started out skeptical of the "home plate" area, but as the game went on I started to see the merit of that zone as the primary scoring area. That said, I wasn’t strict about it. If a shot was a foot outside the dots or two or three feet from the top of the circles, depending on the ‘setup’ of the ice, I counted it. I also counted shots from way outside if the passing sequence led to the defense or goalie being off balance or out of position. Finally, I decided beforehand that I was going to look for situations where the puck was floating around in a dangerous area, e.g., in or around the crease. If an offensive player had a reasonable chance to play the puck, even if they whiffed or had their stick tied up, I counted it under the theory that the more the puck is in that area, the more likely the defense is to give up a dirty goal. If no one was nearby, I didn’t count it.

D’oh: During the Penguins’ first-period power play (around 3:50 left in the first), there was a sequence where Malkin took two one-timers from near the top of the right faceoff circle. I counted one as a scoring chance, but I didn’t count the other. The first shot was from slightly further away and from a worse angle. Moreover, the pass took longer to get to Malkin, which let Neuvirth get set. The second shot was from a better angle, and the pass got to Malkin’s stick much faster - as a result, Neuvirth didn’t come over square. Although Malkin put the shot high and wide, had he put it on net it had a great chance of going in. That sums up how I went about this exercise. I tried to take a holistic view of the play, not just the distance and angle of the shot relative to the "home-plate area." Several big factors for me were: Was there effective traffic - not just players around the net, but unguarded players with their sticks ready for deflections or rebounds? How much time and space did a player have? Was there someone right on him? Was the player on their forehand or backhand? Did the player shoot right-handed or left-handed, and how did this affect the angle of their shot? Was the goalie in good position? Was the pass playable? There was a play in the first where Chimera threw a pass to the far post for Alex Semin. Had Semin got a stick on it, it was likely a goal. Nevertheless, the pass was a couple feet in front of Semin and he had no chance to get his stick on it - thus it wasn’t a scoring chance by my standards. Part of my standard was that "it takes two to tango;" in other words, you had to have both a good pass and a reasonable chance of reception. A terrible pass to a wide-open player wouldn’t cut it, nor would a great pass to a player whose stick was tied up.

Q3. One point touched on in response to that last question - should the shooter have any impact on whether something is a scoring chance or not?

RP: I definitely believe that the shooter matters, and the type of shot matters. One chance that I had noted as questionable (but didn’t count as a scoring chance) was a Mike Knuble backhand from the slot. He had defensive pressure and couldn’t get a lot on the shot. Geographically that’s in the heart of home plate, but he didn’t really get a clean shot off and it was on the backhand. Add in that Knuble isn’t known for his shooting prowess and I just couldn’t count that as a legitimate scoring chance. On the other hand, J.P. talked about how Ovechkin’s goal wasn’t a scoring chance because of where he shot from. I counted that as a scoring chance, but most shooters probably would not qualify from there. It makes no sense to me to treat shots from the same location as equal if the two players taking the shots are not equal (and this is really just a variation on what J.P. discussed above regarding not all scoring chances being equal). You can’t say that a shot by Ovechkin or Alexander Semin is the same kind of chance as a shot from Karl Alzner or Jeff Schultz from the same location (though a shot from the press box probably isn’t a scoring chance regardless). Logically that just makes sense. And if you take it a step further we can see that evaluating scoring chances by considering the shooter makes even more sense because it aligns with how the game is played. Coaches specifically put systems and checking lines together to limit certain players. A breakdown that allows Ovechkin or Stamkos a chance to get an open look is the kind of breakdown that makes coaches pull their hair out: that’s exactly the kind of play you want to prevent. A breakdown that allows a fourth liner to take a quality shot is not going to be nearly as aggravating (though obviously is not something the coaches want to see). Think of the Dmitri Orlov breakdown from a few weeks back, those were quality chances but they were 4th liners. I don’t think you can treat those chances the same as if they came against Neal and Malkin (and that’s probably why Orlov doesn’t see much ice against the Neals and Malkins of the league). In short, the shooter is a part of the breakdown, a part of the mistake. Without the shooter, you have an incomplete picture of the play as a whole and thus an incomplete picture of the putative scoring chance.

JP: I guess it was a bit of a trick question on my part. On the one hand, of course the shooter matters. That’s just common sense. But on the other hand, if you’re going to base what you consider a scoring chance on who’s shooting (and other factors), how useful can the data be to anyone else? It gets back to grading chances. I’m sure you could track chances in a manner wherein you’re taking the spot on the ice (objective), the shooter (subjective) and mitigating factors such as defenders, shot type, etc. (subjective). But again, the more subjectivity you introduce the less useful the data becomes to a larger community of would-be analysts. Essentially, what Rob’s describing probably mirrors what teams are doing internally, and what I’m talking about is how the rest of us are doing it. And that’s another major point - there’s no way that teams treat all shot attempts from inside the "home plate" area the same.

SME: The shooter definitely matters for all of the reasons that Rob mentioned. Teams are not going to play all players equally, and that is going to create opportunities that otherwise wouldn’t exist. I highly doubt any coach thinks his team is doing its job if it allows Malkin as many one timers from the dots as he got in the game. However, if they’re stopping Malkin from getting those, I’d have to think the coach is happy with that, even if they’re giving up that shot to someone like Adams. How many times have players like Malkin or Ovechkin scored (or rang the post) on shots that other players wouldn’t have even attempted? It makes it far more difficult to create a method of tracking that is highly useful across teams, but it would also be inaccurate to treat all shooters as equal. It’s a cruel reality of tracking scoring chances. Hockey isn’t a sport composed of events that are so easily separated from one another, like baseball. However, I still think there is utility in the ratio to show who had greater control in a game.

GOULD: I tended not to pay attention to shooter -- at least not consciously. Since my standard was basically "have I seen non-fluky goals scored that way," I didn’t care who was driving the play, only what was happening. Honestly, it didn’t occur to me to take the shooter into account. Would I do it differently next time? I don’t know -- I’m as curious as the rest of you to read the other responses to this question. But I do worry that depending on shooter could become recursive in a bad way. "Player A is a great player because he has more scoring chances. Therefore, I will rate plays as scoring chances when they are made by Player A which I wouldn't for other players."

renstar: I think the shooter should definitely matter. That said, I had enough trouble deciding whether the situation should result in a counted chance and never even considered the shooter in my determination. Even with tape-delay and rewind, the task is difficult enough. Tracking the chances might have taken me an additional 30 minutes if I had looked out for the shooter (it took at least a couple of hours already). I think it would be near impossible to do live.

D’oh: Something that really vexed me was whether or not to change my judgement of a scoring chance based on which players were involved. At 17:38 of the third period, there was a scrum in front of the Caps’ net. The Pens had Neal and Kunitz there, both of whom I think are pretty good around the net. I scored it as a chance, but had that been Park and Vitale, I might not have. In the end, I came down pretty much exactly the way Rob did - I even recall going through the exact same thought process during both the Ovechkin goal and the Knuble backhand, and I scored them the same way (chance and not a chance, respectively). At the same time, I agree with JP - judging things by the shooter introduces another element of subjectivity to the process, and it’s another reason why I’m now less comfortable with the validity of scoring chance data than I might have been before.

Below is a table with each of our final results for each team during the game:

Scorer	Caps	Pens	Total
Rob	10	21	31
J.P.	10	23	33
SME	15	25	40
Renstar	19	26	45
GOULD	34	49	83
D'oh	11	16	27

We also have a chart that shows how each player performed in the eyes of each scorer, with a column for the mean on the far right:

Player	Rob	J.P.	SME	Renstar	GOULD	D'oh	Mean
Erskine	-3	-5	-3	-4	-5	-3	-3.83
Wideman	-2	0	2	0	-5	0	-0.83
Ovechkin	-2	0	4	3	1	4	1.67
Halpern	-3	-2	-1	1	-2	-1	-1.33
Brouwer	-3	-6	-4	-5	-6	-4	-4.67
Laich	-4	-6	-3	-2	-6	-4	-4.17
Knuble	-1	-3	0	0	0	0	-0.67
Chimera	0	-1	1	2	0	0	0.33
Hendricks	-4	-7	-5	-5	-6	-5	-5.33
Alzner	-3	-2	0	-3	-9	-1	-3
Semin	-1	2	4	7	0	4	2.67
Ward	-4	-5	-4	-4	-5	-4	-4.33
Hamrlik	-4	-7	-3	0	-2	-3	-3.17
Eakin	-4	-5	-4	-4	-6	-4	-4.5
Carlson	-5	-7	-4	1	-2	-2	-3.17
Orlov	-6	-7	-4	-4	-5	-4	-5
Beagle	-4	-5	-4	-4	-7	-4	-4.67
Perreault	-2	0	2	0	0	1	0.17

Tomorrow, in Part II, we'll look at some specific examples and demonstrate how various factors resulted in our specific decisions.

Ask the Owner: Ted Leonsis Answers Your Questions (Part II)

If you need set-up, here's Part I of the interview. Otherwise, let's dive right in...

6. Caps fans see their team currently well below the salary cap, a top skill player likely to leave in Alexander Semin [Ed. note: obviously this came to pass], and (with all due respect to Wojtek Wolski and Jack Hillen) no significant off-season upgrades via unrestricted free agency (though the team did take on salary in Mike Ribeiro). During the same summer, the Wizards have used the NBA's amnety provision to waive Andray Blatche, which will cost the team $23 million through the 2014-15 NBA season. Is there ever a point at which one team's financial situation or obligations will impact the other team's spending on player salaries, or are the two budgets wholly independent?

Leonsis: I wouldn't downplay the trade for Mike Ribeiro; he had 63 points (18 goals and 45 assists) in 74 games last year and was a strong set-up man. That point total would have been the second highest on the Capitals last year and only two off Ovechkin's team-leading 65. He has scored 50 or more points the last eight years. Therefore, we expect him to play an important role for us this season.

But back to your question, the Capitals and the Wizards are run fully independently. They have separate budgets, profit and loss statements, salary-cap management systems and CBA rules governing them.

We are financially committed to both teams, and the Capitals have spent near the salary-cap ceiling. The idea is to spend money wisely, not just spend. Our combined payrolls may top more than $125 million next year - so we are committed to both teams.

7. With the return of Adam Oates, Calle Johansson and Tim Hunter to the organization coming on the heels of Dale Hunter's and Olie Kolzig's return (all in various coaching capacities), that's quite a run on alumni from the 1990s. Is that a conscious decision - is there something those Caps teams had that the current one lacks - or is it just a coincidence? If the organization needs a bit of a cultural change, is it possible without bringing in fresh(er) voices?

Leonsis: Simplified, our goal was to identify and hire the best coach for the Washington Capitals, not the best former Capitals player. We took our time, looked for the best qualified person, spoke to a number of people and interviewed several candidates.

Adam Oates was our top choice. He was an assistant with the Eastern Conference's best team last year, one that competed for the Stanley Cup. He is a Hall of Fame inductee. Adam was our captain and always was viewed as an intelligent player, a coach on the ice and the quarterback of the power play. Dick Patrick and George McPhee felt extremely comfortable with him, and I respect their judgment. The feedback I have received so far has been overwhelmingly positive.

Yes, Calle Johansson was a great Capitals player too; he still holds our record for most games played in franchise history as well as the mark for most points by a Caps defenseman. He also served as an alternate captain and was looked to for his leadership qualities. He has stayed connected with our organization since retirement, and we felt he would be a great fit for our current crop of defensemen.

Tim Hunter played more than 800 games in the NHL and coached another 1,000. In addition to being the best goalie in team history, Olie Kolzig was a respected leader in the locker room and was viewed as a de facto captain. As the associate goaltending coach, Olie's primary responsibilities center on working with all of the goalies in our system.

But all of these "voices" are new to our players. As I have said, sometimes too much change concerns me. Coaches and players need time to get to know one another and gel. Adam is a strong communicator, and we have a solid group of leaders in our locker room, so I'm hopeful that process will be relatively short.

8. Two of your three coaches on the bench (Oates and Johansson) are NHL "rookies" in their respective current roles. Meanwhile, you've seen at least three significant veterans leave this offseason (Knuble, Halpern, Vokoun). Is there enough leadership on this team on the ice and "in the room," and where is it?

Leonsis: Yes, all of those players were veterans and provided a leadership voice for us, but I would say less so during the playoffs. First of all, leadership on any team and in any organization does not fall to just one person. We have numerous leaders - some do so verbally and others exemplify leadership qualities through their actions.

It obviously starts with Ovechkin, but it certainly doesn't end there. Players such Backstrom, Brouwer, Green and Laich have had the alternate's "A" sewn on their jersey, but I also see leadership from Alzner, Carlson, Chimera, Hamrlik, Hendricks and Ward. Leadership has many faces.

9. What is your role in hockey operations today (from player acquisitions to coaching hires and so on), and has that changed in any way since you took over as owner? How much input would you have when it comes to making offers to $100-million free agents (like Parise/Suter/Weber) or trading a marquee player or hiring head coaches?"

Leonsis: If it involves $100 million, you can be assured I'm involved -- any owner would be. But I am less and less involved in the day-to-day hockey operations than you might imagine. I do speak with Dick Patrick and George McPhee on a near-daily basis, but that usually is in the form of an update.

Where I am more hands-on is developing an organizational philosophy, a vision, a plan and strategies to execute that plan. I'm also involved in creating a team and front-office culture and a responsible budget.

For the record, the Capitals have not made free-agent offers that are back-end loaded. Our contracts with Ovechkin and Backstrom are straight-forward agreements. I was involved with the Ovechkins when we signed Alex, but I trust the expertise and advice of our president and GM. Their role is to run the hockey operations department and execute a plan we have agreed upon.

If we were to trade one of our marquee players, I would have to hear the pros and cons for such a deal, and I would be interested in the perspectives of the president, GM and coach.

This year I wasn't able to attend the draft and wasn't in the "war room" the first day of free agency. I was kept apprised of the offers we were planning and extending, but this was just within our ordinary course of communication and business.

I'm not an expert on the draft, free agency or making trades. I'm not an amateur scout, professional talent evaluator or a coach. We hire experts in those areas, and it is important to allow them to do their jobs. I know my limitations. Sometimes I wonder: Do our small, dedicated group of fans on message boards ever have that level of self-awareness?

10. Finally, the game is always changing, and recent trends seem to have it headed back to mid-90s levels of obstruction and scoring. Do you like where the game is and where it's heading, and do you feel that the Caps have been agile enough to adjust to the changes as they've come?

Leonsis: With professional sports it seems you are always changing in some fashion or another. There is something new employed by a team and coaches and players react to it. In the end, however, it comes down to a team buying into a system, executing it and sometimes having a bit of luck.

I love our game. Our fans love our game. Are there areas for improvement? Absolutely there are. That is the nature of business in general. When you become satisfied, you become stagnant.

So, yes, I think we have made adjustments, and I'm sure you will see changes made by Adam Oates and his staff. We will continue to make adjustments in order to win a Stanley Cup.

Furthermore, I am fortunate to be part of the NHL's executive committee, and I am involved with a small circle of other owners in all aspects of the CBA discussions and potential rule changes. So we are well aware of what is on the horizon.

Thanks again to Mr. Leonsis for taking the questions and to you all for helping craft them. Let's do it again some time, shall we?