How do computer vision and machine learning boost the world’s leading fully automated real-time sports analytics? Are you interested in how we introduced these technologies to our tech stack?
To learn all about it, you can watch the video above or read the article below!
Senior Software Developer Santeri Karjalainen is a backend developer specializing in computer vision systems and machine learning for Wisesport. Here he provides professional insights into the analytics platform.
A smart tech stack results in scalability
“The Wisesport platform involves many aspects of information processing. The core of the system is built on collecting spatiotemporal data and calculating many different statistics. We use a radio-based tracking system to track players, referees, and the puck or ball. Advanced algorithms are run in the cloud to refine the spatial data and to calculate statistics for players and teams. We also use multiple cameras to record and analyze the match.
The Wisesport platform serves all data to end-users and third parties via different APIs and frontend applications. Additionally, our platform provides new experiences for spectators at sports venues.
Early on, Wisesport made good choices regarding the technology stack. We selected the microservices pattern and designed everything for horizontal scaling. This gives Wisesport a high degree of scalability in both the number of clients we can serve and the variety of features we can offer.”
A new take on computer vision in sports analytics
“Computer vision systems have long been used to track humans and other moving objects with high precision and accuracy. That is why we introduced computer vision to Wisesport with very promising results.
We have trained our human tracking model specifically to track hockey and football players from multiple different angles. Tracking a regular football is also possible with a separate model.
Detecting players and referees from a camera feed in real time is fairly easy with today’s technology. However, identification of players is much more difficult. Robust tracking of humans over time is complicated, especially when other humans sometimes obstruct them. This usually requires many different camera angles specifically set for that purpose. Processing all this requires a lot of compute.”
The Wisesport platform allows tracking players in multiple ways
“With the Wisesport platform, we use radio-based tracking data to uniquely identify players in a camera’s field of view. That is a fairly straightforward mapping from the 3D object space to the camera space. We then augment the tracking data with all insights provided by the computer vision. These insights include information such as whether a shot was a wrist shot or a slap shot, which way the player is facing, what are they doing with the stick, etc.
Even with just one camera angle, we can estimate the entire pose of players, including their hands, feet, and so on. To push our analytics even further, we build neural networks that infer all kinds of information from the players on the field. It is a great addition to the high-quality data that we already collect.”
“In addition to augmenting radio-based tracking data, the computer vision system produces spatial data for players and referees on its own. This means we can produce accurate spatial data for our analysis if some players or even an entire team is not wearing any radio transmitters.
We use a massive amount of radio-based tracking data for supervised learning, which allows us to approach computer vision based identification with an edge that no one else has. With the tracking data, we could build a model that can identify players with fewer camera angles than has previously been possible.”
Now we provide even more precise stats and deeper insights
“Our analytics system is agnostic regarding the tracking system. This means that we can separately improve the actual raw spatial input to the analytics. Any improvement in the quality, complexity, and richness of the model of the sporting event propagates into even more precise statistics and deeper insights.
Using just computer vision to track players and match events enables running our match analytics without any wearables. A well-known drawback of standalone computer vision systems is that they require a lot of computing power either locally or in the cloud. However, we have the data and expertise to build the most efficient solution to this problem.”
“Alongside applying neural networks for object recognition and different regression problems, we also experiment with end-to-end neural network architectures that ingest tracking data. The amount of data that we have collected so far allows us to fit models that seem to understand the flow of an ice hockey match and how players behave in different situations.
For example, we can infer the position of the puck with high accuracy with a deep learning system that takes only the positions of players and referees as inputs. Likewise, if we remove a player from the data, the neural network can give incredibly accurate predictions of where the player is. Since we already have a massive amount of data from different levels of ice hockey, we can dig deep into how different leagues and teams play.”
What sets Wisesport apart
“In general, I find applying these advanced machine learning methods such as autoencoders and both supervised and unsupervised learning very exciting. It will drive the field of sports analytics into a new age, especially with the data we have at our disposal.
Neural network architectures similar to what we have experimented with have been used and studied before. However, the amount of data to train these networks is usually very limited in its modality. The richness of data is what sets Wisesport apart from the rest.”
Stay tuned for more in-depth features on Wisesport, computer vision and machine learning!
