Slashdot Mirror
Casting Doubt On the Hawkeye Ball-Calling System
Human judgment by referees is increasingly being supplemented (and sometimes overridden) by computerized observation systems. nuke-alwin writes "It is obvious that any model is only as accurate as the data in it, and technologies such as Hawkeye can never remove all doubt about the position of a ball. Wimbledon appears to accept the Hawkeye prediction as absolute, but researchers at Cardiff University will soon publish a paper disputing the accuracy of the system."
And ultra-accurate GPS like system that tracks the position of balls in nanosecond detail. They can call it Your Object Universal Remote Movement Observance Mechanism, or YOUR MOM for short.
Hawkeye and the like deliver a consistent result. It matters not at all if the ball is in by two Centimetres but is called out, provided that error is consistent throughtout the match.
If both players, or teams, are playing by the same margin of error, the contest is fair.
In cricket for instance, I would accept the computers call over umpires any day of the week!
Why not use a radio transmitter in the tennis ball (or soccer ball or whatever) to record its exact position? I am certain this has been discussed and I wouldn't be surprised if it's already in use. The article's "Hawkeye" just works by optical analysis.
It's been tried in soccer. The latest attempts were prior to the last couple of World Cups IIRC, but the systems were plagued with problems, not the least of which was the survival of the transmitter.
http://www.gizmag.com/go/2790/
"Anyone that has ever gotten an idea based on any of my work and done something better with it-good for you."--J.Carmack
Further to that, if the transmitter can't survive in a soccer ball (where a well-struck shot probably moves around 120-130 kph) then there's no way it will handle travelling over 200 kph after a serve, followed by a (at least) 100 kph forehand return (a net >-300 kph in a fraction of a second!).
Also, a radio transmitter cannot account for the distortion of a ball upon impact, which will depend on velocity, angle of rotation, angle of impact, surface being played on, etc etc etc...
Triangulation of radio signals is not accurate enough to give sub-centimeter accuracy and the added mass to the tennis ball would probably cause the players to have some objection to adding a radio transmitter into the ball.
The claim that the Hawkeye system gives an average of about four millimetres of error seems somewhat reasonable, given that we're getting accuracy greater better than two centimetres on detecting objects with a single camera with optics as large as the last segment of a typical pinky. (FWIW, here's a short demo of what we're working on for our autonomous underwater vehicle)
However, the suggestion to display the error range for a particular shot and leaving the final decision to a human from TFA is quite reasonable and is how it should be. Blindingly trusting technology or discarding it altogether is unreasonable.
Jean-Francois Im's blog
I still don't understand why there isn't more research on developing a surface for the out-of-bounds area that temporarily registers the exact impression of any impact on it.
I envision something that looks like a big LCD touch screen (but more durable). Every time something made contact with the active surface, a record of the ball's "footprint" could be recorded (and even temporarily displayed wherever it touched the surface). That would allow for highly precise measurement of the ball's landing position, and it wouldn't need to incorporate any new materials into the ball itself. The active surface would only need to be in the out of bounds area, and even then, it would only need to be half a foot wide in order to cover the important zone where the ball's landing position is questionable.
A system such as Hawkeye CANNOT BE MORE ACCURATE than humans. From the link in the article, the Hawkeye system uses 5 cameras to compute the 3D position of the ball. That's an overdetermined system of equations, which cannot have a unique solution due to observation errors in the camera views.
Luckily there's a 100+ year old discipline called statistics, and 60+ years of literature on tracking to help you out in these cases.
So Hawkeye has to complement the equations with an ARBITRARY rule, eg least squares and this arbitrariness makes the Hawkeye estimate neither more accurate nor less accurate than humans, just different. FYI, there are plenty of other arbitrary rules that work, eg least absolute errors, maximum entropy, etc.
While I can't speak for the designers of the Hawkeye, in tracking there are very good reasons to choose one form of error minimization versus another. It only seems arbitrary because you are not informed on the subject, but there's plenty of free papers out there to read and discover.
To explain current methods, please start out with this paper, in particular Figure 2, you'll see that the sort of errors you get from a camera are indeed well fit by a Gaussian. While a camera's perspective transformation is not purely linear (and various forms of distortion make it also non-linear), a good camera with a decent lens estimating the ball location within a limited area is well approximated by a linear model (and you can characterize just how much the error is). Now, a bunch of cameras with a Gaussian error distribution in the image plane with a linear projection out into the world is still a Gaussian (with a transformed covariance matrix). You can then multiply the independent measurements from multiple cameras to get a better estimate. Add a time series to that and apply this recursively and you get a Kalman filter, something invented for aerial tracking and still in widespread use today. If something is good enough for missiles to intercept other missiles, it ought to be good enough for a tennis match.
If the linear approximation not good enough for you, you can use a Rao-Blackwellized Kalman filter. If that's still not good enough because you want to use another error distribution or non-linearizable dynamics, set up a particle filter with a whole lot of particles and enough CPU to simulate it. The point is that what you call arbitrary is a well studied field which is many decades old. You'd be best served by learning about it first before you cast away all that work. I'm not a "tracking" person, just a user of there work. When a field of science has done its job well enough that it has become common engineering, and you can go look up whatever you need in books, with all the derivations, caveats and tradeoffs laid out there for you to see, I would say that that field has done a pretty good job.
The whole media story around this paper is ridiculous. It's a paper from a social sciences department about how the public does not understand the fallibility of these machines due to noise. That's all this paper is about: Hawkeye has error. I hate to break it to the uninformed, but all measurement systems have error. From Galileo to Gravity Probe B, your results can only be as accurate as your measurements, calculations, and statistical models will allow. You can decrease error with various methods, but you can never completely eliminate it. People should not be able to get out of high school without understanding accuracy on measurements, and some rudimentary statistics, but unfortunately our education system hasn't been able to reach that goal. As a result, the public doesn't understand error, and might come to believ
Yes, some people also want to use Hawkeye for some decisions in cricket, the sport that first used it. However the margin of error is far greater (approximately +- 2 inches) in cricket as the cameras have to be a lot further away due to the size of the pitch.
Also Hawkeye finds it hard to pick up swinging, seaming and spinning balls. Basically anything that deviates off its theoretical trajectory either in the air or off the playing surface. Both of which are vital in the LBW decisions where the TV companies and doubtless the Hawkeye people would want to see it used.
Obviously cricket is a far more useful game than tennis so does this answer your question?
another Roadkill on the Information Superhighway
That's compatibility for legacy (analog) shopping carts with the one wheel stuck sideways.