I’ve been tinkering for a couple of years with applying computer vision to broadcast footage of soccer games. Aside from sourcing the video (for a later post), one of the first challenges I faced is establishing where the camera is looking.
Purpose-built, dual-lens, “field cameras” such as Veo and Trace capture a nearly 180-degree panoramic view. In researching the Veo, I found that it locates the four corners of the field, which provide reference geometry to map between pixel space and world space. The broadcast footage I’m focused on also comes from dual camera setups (primarily Pixellot), but I only have access to the rendered footage that is sent over the wire. This means I only see a slice of the field at a time and have to work out which part of the field the camera is showing.
One solution to this is to register key points that a detector can find and use to locate the camera’s view within a range of potential views. Doing this automatically may prove possible, but to get started I’ve registered a handful of venues by hand. Most venues require labeling or annotating at least three representative frames—typically views of the left goal, midfield, and right goal- adding left-mid and right-mid views often improves coverage. Those labeled frames form the initial backbone of our venue registration system.
A homography maps the field plane to the image. These stadium cameras are typically mounted at press box height, well above and set back from the field. Estimating the physical camera position requires known field geometry (this is where the American football markings help as they’re measurable and consistent between venues) and a constrained camera model. From there, we’re able to derive new homographies by matching general image features.
And, once you have registered a venue, you’ve unlocked the pixel-to-world relationship and taken a giant first step on the long road to auto-analyzing broadcast footage =)
