Skild AI's S1 Robot Learns Soccer through Innovations in Self-Play

Skild AI has achieved a notable advancement in physical AI through the self-play learning of its S1 robot, enabling it to play soccer effectively. This development demonstrates how a strong foundational model like S1 can master complex and dynamic activities by competing against itself in simulations.

Introduced as a flagship model last month, the S1 employs a learning methodology reminiscent of language models, absorbing tasks from in-context demonstrations. However, since S1 models rely on human training data, they are intrinsically limited to human-like capabilities.

Skild AI believes that advancing physical self-play can enable robots to enhance their skills beyond human potential. According to them, "we believe physical self-play will enable robots to far exceed human capability."

The Self-Play Era

The concept of self-play has been pivotal in advancing artificial intelligence, predating the advent of language models. For instance, AlphaGo’s self-play approach led to its historic victory over world champion Lee Sedol in 2016, achieving extraordinary success in navigating the complexities of the game.

This technique saw its full potential when extended to multi-player environments in games like StarCraft II and Dota 2, showcasing AI systems that developed strategic capabilities absent from their initial training datasets. Although the trend shifted towards reinforcement learning with quantifiable rewards, Skild AI aims to rejuvenate interest in self-play, proposing its potential to ignite developments in physical artificial general intelligence (AGI).

Key achievements in Skild AI’s soccer-playing model include:

• Objective-driven learning: The S1 robot was given the singular goal of scoring, refining its techniques through competition against its earlier iterations without human demonstrations.

• Skill progression: Initially unable to maintain its balance, the robot progressed to mastering dribbling, shielding the ball, tackling, and regaining its footing during play.
• Emerging teamwork: Preliminary tests with four-agent teams illustrated the beginnings of passing and coordinated movements.

Soccer serves as an excellent platform for assessing the robot's capabilities, driving towards the larger goal of enabling robots to learn autonomously without constant human instruction.

A Physical Revival

Skild AI focused the S1 model on a single directive: score. The self-play methodology allowed the robot to learn strategies to accomplish this, concurrently enhancing the skill level of its adversaries. The result was an iterative improvement cycle that turned learning into an increasingly challenging game.

Starting from barely mastering basic movements in NVIDIA’s Isaac Sim, the S1 progressed significantly in simulated months. Over time, it developed advanced capabilities naturally, such as dribbling and tactical maneuvers, simply because these skills were advantageous for scoring.

During 140 years of simulated competition, the model demonstrated a successful transfer from virtual gameplay to a physical robot, heralding positive implications for the future of self-play in pursuit of physical AGI.

Why Soccer?

While robot soccer competitions already exist, the performance levels remain below those of human players. Soccer is particularly advantageous as a testing ground for robotics, demanding both physical skill and strategic insight. However, Skild AI plans to extend its self-play methodology to encompass various everyday robotic tasks beyond just sports.

At Scale

The simulation conducted by Skild AI ran for an extended duration with fundamental goals in mind. This spurs the question: what potential arises when simulations are extended significantly in time and complexity, perhaps in settings like construction sites, factories, or homes?

Skild AI hints at the vast possibilities still to be explored in physical self-play, as they plan future developments that will investigate social behaviors arising in larger teams, with applications ranging from collaborative tasks to urban navigation on a grand scale.

The ongoing advancements at Skild AI suggest we are only beginning to scratch the surface of what's achievable through physical self-play in robotics.