Object detection and camera correspondence for soccer games

Abstract

A sensitive cost for the broadcast industry is the employment of many operators for man- aging the production of video content. The automation of video production is the future of the broadcast industry for reducing its cost as well as for enhancing the experience of its costumers. Two of the major steps to understand a game are the localization of the images from the cameras into the soccer field and the description of the image foreground. To do so, a solution for image localization into the soccer field for close-up view cameras has been implemented along with an object detector for soccer. To obtain such localiza- tion, a correspondence has been made between the main camera that can be calibrated and a close-up view camera that can not. Then, by using the concept of transitivity, the content of the close-up view camera can be located in the soccer field. The correspondence between an image from the main camera and one from a close-up view camera is accomplished by the extraction of “very large features” that an object detector outputs. As most of the object detected are players, a patch matching algorithm is used to differentiate them so that the features from both images can be matched. The object detector achieves more than 70 mAP-50 on a private dataset while the patch matching algorithm reaches an accuracy of 90%. The overall procedure predicts an accurate localization of close-up view camera content for 90% of the frames without exploiting the time dimension.