The SEC improved officiating accuracy by nearly 8 percent in 2016 thanks to having more eyes on the replays, SEC officiating coordinator Steve Shaw told CBS Sports.
Last season was the first in which the NCAA let conferences use people other than the stadium replay official to assist on reviews. The SEC had three replay officials at a command center in Birmingham, Alabama, to help the stadium replay official for all reviews. Shaw said he determined that collaboration helped 18 of the 226 reviews produce a correct outcome. The SEC declined to specify Shaw’s methodology for how he evaluated that a correct outcome was due to collaboration.
“In anybody’s business, if you can get an 8-percent increase in productivity, that’s huge,” Shaw said. “We’re very pleased with it. We think we got better as the year went on.”
The NCAA Football Rules Committee is expected to continue collaborative replay as an experiment in 2017. Collaborative replay is clearly here to stay. The longterm question is picking the best model if the conferences ever elect to follow the same procedures.
“I don’t think we have enough data yet [for uniform guidelines],” national officiating coordinator Rogers Redding said. “That’s another reason for extending the experiment for another year, to get a better sense of what’s best.”
The SEC and ACC used a centralized command center in 2016. The Big Ten studied, but didn’t use, centralized replay. Instead, the league relied on the referee looking on a tablet and collaborating with the on-site replay official. The Big Ten will use that model again in 2017.
“My best people are head referees,” said Big Ten officiating coordinator Bill Carollo. “That’s why they get promoted. That’s why they get paid a little more money. Using the referee allows better communication to a coach on why a call stands or is overturned.”
As an experiment in 12 games, the Big Ten used replay officials from a command center to make calls that didn’t count. In a very small sample size, two of the 30 reviews in those games would have resulted in a different call than the replay official and referee actually made, Carollo said.
One of the different calls resulted in the ejection of Penn State linebacker Brandon Smith for targeting against Michigan. The Big Ten announced after the game that targeting shouldn’t have been called.
“One of my top referees went over there and said, ‘We can confirm it, it’s targeting,’” Carollo said. “They went too fast. I said, ‘Guys, you have to actually see the shot that says you can confirm it or the shot that says you’re going to overturn this.’ The ref trusted his guy a little too much up in the booth. That was a big one that another set of eyes would have caught.”
The ACC had 30 percent of its reviewable plays reversed in 2016, up from 24 percent a year earlier. ACC officiating coordinator Dennis Hennigan said no conclusions can be drawn from that change, and he doesn’t have a way to quantify how much collaborative replay helped.
“It’s adding another set of eyes to the replay,” Hennigan said. “We were very happy with the way it worked out.”
The SEC had almost 43 percent of its reviewable plays reversed last season, a significant spike from 37 percent in 2015. That’s the SEC’s biggest one-year increase for reversals since 2010.
It’s important to remember collaborative replay simply passes the subjective question about a call on the field to other people. While more eyes may help, it doesn’t guarantee the correct answer.
“It was not perfection because perfection is in the eye of the beholder,” Shaw said. “You’re never going to solve gray [area] plays. If we had our coaches at the table, they might vote 7-7 on a gray play. The goal is to solve incorrect outcomes.”
With more eyes to help, the SEC had only slightly more play stoppages last season. The average review length increased by six seconds to 1:22. The average times during the first two weeks exceeded 1:40.
“We had to get a handle on that,” Shaw said. “I think at first, and it’s only human nature, the replay guy in the stadium was a little concerned about Big Brother looking over his shoulder. But it’s not just Big Brother telling you the answer. It’s a collaborative process. As we progressed through the season, our replay guys in the stadium liked the collaboration.”
| Conference | Stoppages Per Game | Reviews Reversed | Average Review Time |
|---|---|---|---|
| SEC | 2.23 | 42.90% | 1:26 |
| Pac-12 | 2.43 | 39.70% | 1:01 |
| AAC | 1.84 | 39.60% | 1:16 |
| Sun Belt | 1.71 | 37.50% | 1:28 |
| Big 12 | 2.61 | 34.40% | 1:37 |
| Mountain West | 2.04 | 33.10% | 1:49 |
| Big Ten | 2.23 | 30.90% | 1:11 |
| ACC | 2.46 | 30.30% | 1:21 |
| MAC | 2.85 | 29.30% | 1:03 |
| C-USA | 1.41 | 26.30% | 1:25 |
| 2016 Average | 2.18 | 34.50% | 1:20 |
The Pac-12 experimented with a command center in 2016 to monitor replays only for Oregon and California conference games. No decision has been made yet on whether the Pac-12 will use collaborative replay full-time in 2017, league officiating coordinator David Coleman said.
“It was a good experience for us,” Coleman said. “It gave us an opportunity to advise and consult and make sure our replay staff in those two locations was considering everything they needed to get a call right. We see the possibility of it growing in the future. Obviously, there are costs involved. That has to be considered.”
The Big 12, which was the only Power Five conference not using collaborative replay in 2016, will adopt centralized replay next season. Big 12 commissioner Bob Bowlsby said the league intentionally waited a year to monitor the experiments by other conferences.
The Big 12 is budgeting about $1 million to lease space to build a command center in Irving, Texas, where the Big 12 office is located. Some of that cost is due to building a small studio unrelated to replay, Bowlsby said.
Big 12 replay officials worked the most controversial game of 2016. Oklahoma State lost to Central Michigan on a Hail Mary and lateral with no time left after an untimed down. By rule, the MAC on-field officiating crew and Big 12 replay officials should not have allowed the play to happen. The MAC and Big 12 officials were suspended two games.
That finish is “exactly the type of play you’re hoping [collaborative replay] will help,” Big 12 officiating coordinator Walt Anderson said. “Let’s get a few more people and maybe somebody else will have an idea of that rule. We’re training a little different this offseason so the expectation is replay officials are aware of every penalty enforcement on the field.”
Unlike the SEC, which intentionally keeps Shaw away from collaborating on replay calls due to conspiracy theorists, the Big 12 will use Anderson and assistant officiating coordinator David Warden. If a call is difficult to determine, Anderson or Warden will make the final decision. That’s an approach used by the NFL, whose top officiating executives review plays.
“I think it lends itself to more credibility,” Anderson said. “It’s not that I don’t make mistakes. I make mistakes because I’m a human. But nobody spends as much time studying this game from an officiating standpoint than I do.”
As for conspiracy theorists who might question the officiating coordinator weighing in on replay calls, Anderson laughed and said he’s used to those fans.
“I think most of them would say, whether Walt’s involved or not, they still think it’s a conspiracy,” Anderson said. “We’re always favoring Texas or we’re always favoring Oklahoma. God, whoever is the highest-seeded team, that’s the one [fans think] they’re going to make sure wins. That even exists for the officials on the field making decisions in real time. They think they’re conspiring or people are betting on the game. That [thinking from fans] is going to exist. The bottom line is just get the play right.”
In the ACC, Hennigan wasn’t one of the two or three replay officials communicating with the stadium replay official. “Certainly, if I’m in the video center and I think an incorrect decision is being made, I will get involved,” he said. “I’m not going to sit back and let what I think is a mistake go uncorrected.”
The American Athletic Conference plans to use collaborative replay for the first time in 2017. The league hasn’t decided on its model, though it’s leaning toward the Big Ten approach with the referee assisting via a tablet.
“I really like the game to be worked on the field at the stadium by the officials we assign,” AAC officiating coordinator Terry McAulay said. “That’s not to say the other way is bad.”
But Carollo expressed concerns that command centers located in conference offices create conflicts of interest.
“I don’t like the structure of a collaborative center down the hallway from the commissioner because the conference may have something to gain if a certain team wins or loses – money-wise, playoff-wise, bowl-wise,” Carollo said. “Of course the conference wants certain teams to win. Conferences don’t make calls, but there is some pressure. That’s why we separate our officials away from the conference office. I want neutrality. That’s what the coaches want.”
Said Hennigan: “I didn’t see any of that pressure [in the ACC command center]. One thing officials have, whether they’re on the field or in the replay booth, is integrity. A good official is not affected by outside pressure.”
Carollo envisions a day when all Power Five or FBS replay calls come from one or more NCAA command centers. There would be no conference affiliation attached to these centers. The replay officials would all be trained the same way for consistency, an approach that’s now possible since College Football Officiating, LLC made Jim Blackwood the first national replay director.
“Why does every conference have to spend a million dollars for collaboration?” Carollo asked. “I’m not sure the MAC or Mountain West or Sun Belt all need their own centers. I don’t think they can afford it, and I don’t like the makeup.”
| FBS Season | Stoppages Per Game | Reviews Reversed | Average Review Time |
|---|---|---|---|
| 2016 | 2.18 | 34.50% | 1:20 |
| 2015 | 2.14 | 31.60% | 1:17 |
| 2014 | 1.94 | 28.40% | 1:15 |
| 2013 | 1.81 | 29.30% | 1:14 |
| 2012 | 1.73 | 28.20% | 1:20 |
| 2011 | 1.63 | 28.30% | 1:22 |
| 2010 | 1.68 | 23.80% | 1:21 |
| 2009 | 1.7 | 26.80% | 1:20 |
| 2008 | 1.55 | 24.60% | 1:20 |
| 2007 | 1.48 | 26.20% | 1:45 |
| 2006 | 1.44 | 25.10% | 1:49 |
| 2005 | 1.26 | 30.20% | 1:55 |
Source: NCAA