Suspicious use of the hawkeye system

[SnowWhite]

In the video in the second game at the score of 13-16, a lift is called out at the back line, in the back left corner. It's very close and the call is challenged. After a lengthy wait the hawkeye system shows the animation of what seems to be a smash going long by a mile at the back line down the middle of the court.

It happens at the 40 minute mark.

What is this? Malicious intent? Technical difficulties? Or just incompetence?

Surely if this happened at a more critical moment in the match or if it was a more high stakes game (even though this is a world tour QF) this couldn't just go undetected?

Hawkeye is kind of a niche subject, so sorry if this thread isn't in the right location.

 

[stradrider]

So... your first assumption is... malicious intent? Perhaps you think that most likely explanation "they" were bribed? . And of course, if it's not technical error than it MUST be incompetence... Most likely..

 

[phihag]

HawkEye is not perfect. There is a long history of incorrect calls or displays, both in the time when it was manually operated and now that it is automatic. The algorithm may not (correctly) detect the shuttle, somebody may be in front of one or more of the cameras, or there may be some other problem, such as a broken camera, loss of power, network interruption, a flaky cable, etc. .

In particular, in this case a player with a white jersey was very close to the shuttle before it fell to the ground, and this could certainly lead the system to lose track of the shuttle.

How could this even be malicious intent? The call is correct, after all, just a little delayed and displayed strangely.

What makes you think that this went undetected? Certainly the HawkEye technicians noted this. From my experience managing technical systems at tournaments and other events (though not HawkEye), I'd say everybody behind the curtain was scrambling to restore system functionality (in this case primarily making a correct call) as fast as possible.

 

[Littlejohn]

Great response Philhag, I have to say it always irks me when people jump onto any issue and proclaim that it roves' malicious intent or the total failure of a system or process. I wonder how many time a day these people inadvertantly make mistakes.

Having, like you, worked events for many years, from club to olympic level I KNOW how dedicated and honest 99.9% of the officials and volunteers are, how hard people work to quickly to correct any errors or faults.

Yes mistakes are made, sometimes unexplainable ones but in 27 years of volunteering I have seen only a couple of malicious or politcal decisions. So please folks lets not jump to conclusions and start throwing mud when things go wrong, unless of course you have firm conclusive proof rather then kneejerk sensationalist conspiracy theories

 

[SnowWhite]

Please. I never "jumped to conclusions". I merely entertain multiple possible reasons for a mistake in a system I know nothing about.

About it going undetected, I was mostly speaking of the players themselves, who should undoubtedly have realized that the shot displayed on the screen was not the shot they played. I imagine if this happened at 19-19 in a third game there would be some sort of protest, unless they already knew their lift was out.

As for the conclusion of the situation. From watching the rally initially, I'm not so sure that lift was out, though admittedly it's not a great camera angle for us to see.

 

[phihag]

About it going undetected, I was mostly speaking of the players themselves, who should undoubtedly have realized that the shot displayed on the screen was not the shot they played. I imagine if this happened at 19-19 in a third game there would be some sort of protest, unless they already knew their lift was out.

Neither side seemed to think it would gain an advantage by delaying the game. So what's the point of protesting then?

The rules doe not allow overriding HawkEye. Even if they did, the original call was Out anyway, so that would stand.

All you'd get by protesting is further delay of the game, and maybe even a yellow or red card.

 

[SnowWhite]

Neither side seemed to think it would gain an advantage by delaying the game. So what's the point of protesting then?

The rules doe not allow overriding HawkEye. Even if they did, the original call was Out anyway, so that would stand.

All you'd get by protesting is further delay of the game, and maybe even a yellow or red card.

Protest in games is usually ineffectual, but that doesn't stop many players from complaining when a line call or service call or net call is perceived to be wrong. Protest doesn't come from a place of objectivity because the players know it won't make a difference, but in the emotionally tense situation that is a badminton game, emotions can dictate actions, both during play and in between rallies.

 

[stradrider]

This is the best part about the instant review system - players never challenge or protest it because it's automatic and bias is taken out of it as much as possible. So it does it's job quickly and cleanly and does not delay or create controversial discussions that would take away from the game itself, even if something is looking a bit off.

 

[event]

This is the best part about the instant review system - players never challenge or protest it because it's automatic and bias is taken out of it as much as possible. So it does it's job quickly and cleanly and does not delay or create controversial discussions that would take away from the game itself, even if something is looking a bit off.

Until this week, I would have agreed with you 100%. But Wang Yilyu and Ratchanok Intanon both challenged Hawkeye. In the WS final, it was obviously a Hawkeye error. There is no telling whether the players will begin to doubt Hawkeye when it is not so obvious. For example, at 19-13 approximately in the XD SF, it is quite likely that Wang Yilyu only ceased protesting because they were so far ahead. His protests managed to get the fault call reversed, presumably because the umpire asked the service judge for a second opinion, then Wang challenged and was visibly incredulous about the Hawkeye decision. The graphics this week have probably made it clear to players that the system is either flawed or not automatic, or both. But as phihag points out, that doesn't mean they will necessarily stop trusting that the system is coming up with the correct call. We also still have no evidence that - even if someone is behind the curtain watching a video replay and picking a spot on the screen with a mouse - there have been any incorrect calls. There have only been occasional graphics that are thoroughly unconvincing. And despite the protests I cite, even Ratchanok and Wang stopped protesting almost immediately, and Rankireddy and Shetty didn't even flinch at that preposterous graphic so I believe your point about this being the 'best part' is still valid. I hope it remains so.

 

[stradrider]

Until this week, I would have agreed with you 100%. But Wang Yilyu and Ratchanok Intanon both challenged Hawkeye. In the WS final, it was obviously a Hawkeye error. There is no telling whether the players will begin to doubt Hawkeye when it is not so obvious. For example, at 19-13 approximately in the XD SF, it is quite likely that Wang Yilyu only ceased protesting because they were so far ahead. His protests managed to get the fault call reversed, presumably because the umpire asked the service judge for a second opinion, then Wang challenged and was visibly incredulous about the Hawkeye decision. The graphics this week have probably made it clear to players that the system is either flawed or not automatic, or both. But as phihag points out, that doesn't mean they will necessarily stop trusting that the system is coming up with the correct call. We also still have no evidence that - even if someone is behind the curtain watching a video replay and picking a spot on the screen with a mouse - there have been any incorrect calls. There have only been occasional graphics that are thoroughly unconvincing. And despite the protests I cite, even Ratchanok and Wang stopped protesting almost immediately, and Rankireddy and Shetty didn't even flinch at that preposterous graphic so I believe your point about this being the 'best part' is still valid. I hope it remains so.

Yes, very strange... Looks like they were dealing with some crazy glitches there... I think Ratchanok's situation was handles really well. Don't know about Wang's complain, but in my opinion it is likely that it was a correct call and the official that is supposed to control the computers didn't want to change it.

What I liked in the Wang's situation is service judge helping the umpire to take back an incorrect fault. That was well done service judge!

 

[phihag]

For example, at 19-13 approximately in the XD SF, it is quite likely that Wang Yilyu

Again, note that the shuttle is very close to a player in white clothing, so that's probably what's confusing HawkEye here.

The Chen Yu Fei case was totally different though – we saw two results.

The delay used to be that long when HawkEye was manual and vanished when it went automatic. Maybe they've had to revert to manual again?

 

[stradrider]

Again, note that the shuttle is very close to a player in white clothing, so that's probably what's confusing HawkEye here.

The Chen Yu Fei case was totally different though – we saw two results. The delay used to be that long when HawkEye was manual and vanished when it went automatic. Maybe they've had to revert to manual again?

It looks like, as we say, the official responsible to IRS saw that the automatic animation was totally incorrect and corrected the animation by manually selecting the one that is closest to the video replay (only he could see it). Than the referee has contacted him by walkie to confirm that the last replay was the correct one. That was a very good how they have handled it I think.

 

[psyclops]

Followers of HawkEye, please note: white clothing has nothing to do with shuttle tracking by the 10 cameras that are setup by the crew.

The technology requires 6 cameras to reproduce the 'smart replay.' The only time it will get confused is when another shuttle lands at exactly the same time as the rally in question. Rio2016 had two challenge courts, and they had two separate setups of cameras, each with their respective networks.

 

[psyclops]

In the video in the second game at the score of 13-16, a lift is called out at the back line, in the back left corner. It's very close and the call is challenged. After a lengthy wait the hawkeye system shows the animation of what seems to be a smash going long by a mile at the back line down the middle of the court.

It happens at the 40 minute mark.

What is this? Malicious intent? Technical difficulties? Or just incompetence?

Surely if this happened at a more critical moment in the match or if it was a more high stakes game (even though this is a world tour QF) this couldn't just go undetected?

Hawkeye is kind of a niche subject, so sorry if this thread isn't in the right location.

This is the Rules - Tournament Regs - Officiating sub-forum.
So which of these does your subject thread belong to?
If not, you may as well report it inappropriate and request for right location.

 

[event]

This is the Rules - Tournament Regs - Officiating sub-forum.
So which of these does your subject thread belong to?
If not, you may as well report it inappropriate and request for right location.

Okay, interesting question. Is there a sub-forum for officiating technology? Also, as far as I know, the rules changed when Hawkeye was introduced and now it was supposed to be this high-tech system to which players are appealing to challenge the calls. When the call review was first implemented, I believe the rules explicitly stated that a tournament official was going to watch a high-speed video replay and make a decision (which they reported by holding up a paper card in the earliest days). From what I can tell, the question being posed by the original poster is whether Hawkeye is making the call or whether they are reverting to human eyes watching a video replay and then picking a point and generating the graphic. Perhaps phihag can clarify whether this is within the rules as they have stood since Hawkeye was added.

 

[phihag]

Followers of HawkEye, please note: white clothing has nothing to do with shuttle tracking by the 10 cameras that are setup by the crew.

Can you elaborate why? What else is your theory for these failures?

Precisely because the setup uses visible light and 3D computer vision, it will have trouble tracking a white shuttle when it is very close to a white player.

So which of these does your subject thread belong to?

Officiating – after all, HawkEye is part of the officiating setup. Problems with it need to be handled by umpires and referees. If problems persist, BWF may change officiating to have a dedicated IRS umpire review the footage – that's the way it was when HawkEye started.

Tournament organizers also need to know if they can improve anything, so it could fall under tournament regulations as well. And finally, it's not totally out of the question to modify the rules to deal with obviously incorrect HawkEye decisions.

(...) whether Hawkeye is making the call or whether they are reverting to human eyes watching a video replay and then picking a point and generating the graphic. Perhaps phihag can clarify whether this is within the rules as they have stood since Hawkeye was added.

The rules just require an Instant Review System and wisely don't go into details. In particular, any competitor of HawkEye and simple video footage review would be allowed as well. In the beginning (and still as a backup), the decision of the IRS was relayed via three cardboard signs (In/Out/No Decision). This system is still in place, but it is not the backup for the decision making, but for the relaying. After all, the rules don't require pretty graphics. It's just a much better experience for spectators to see pretty graphics.

 

[psyclops]

You go first and provide your basis for the statements you made, such as those listed (i) through (iv):

(i) HawkEye is not perfect.
(ii) There is a long history of incorrect calls or displays, both in the time when it was manually operated and now that it is automatic.
(iii) The algorithm may not (correctly) detect the shuttle, somebody may be in front of one or more of the cameras, or there may be some other problem, such as a broken camera, loss of power, network interruption, a flaky cable, etc. .

In particular, in this case a player with a white jersey was very close to the shuttle before it fell to the ground, and
(iv) this could certainly lead the system to lose track of the shuttle.

There are some on this forum who appear to trust and believe what you write, so the onus, nay, the responsibility is on you to make statements that are not opinions. This is not a medium where I would engage with you on the scientific aspects of the derived graphics.

 

[event]

You go first and provide your basis for the statements you made, such as those listed (i) through (iv): There are some on this forum who appear to trust and believe what you write, so the onus, nay, the responsibility is on you to make statements that are not opinions. This is not a medium where I would engage with you on the scientific aspects of the derived graphics.

I must be missing something. Why is phihag responsible for not giving his opinion? I would say that it is because he is prone to considered opinions that he has gained the trust of readers on the forum. As for what he posted above in this thread, are you seriously asking him to back up the statement that HawkEye is not perfect when even the manufacturers claim only a level of accuracy? I would say what you listed as item (iii) is a very logical conjecture and it would be difficult to imagine correct functionality being maintained in such circumstances. (ii) is an interesting empirical observation and I too would be interested if someone were to be able to produce an exhaustive list of errors but I don't think this statement offhanded or unreliable just because phihag can't produce a catalogue of said errors on demand. If he has been reading and responding to comments on HawkEye and watching extensive coverage, his gut reaction may well be convincing. As for (iv), it is an opinion and I have no idea whether phihag has the technical expertise on the workings of HawkEye to give it as, say, an expert witness but nor did he claim to. It is an opinion based on logic and I see no reason why a reader should interpret a statement saying '[A] could happen' as fact.