Research reveals that AI agents will start building their own society if left alone
As OpenAI CEO Sam Altman said, 'The next big breakthrough is agents,' '
Emergent social conventions and collective bias in LLM populations | Science Advances
https://www.science.org/doi/10.1126/sciadv.adu9368
Groups of AI agents spontaneously form their own social norms without human help, suggests study | EurekAlert!
https://www.eurekalert.org/news-releases/1083323
Researchers from St. George's, City, University of London and IT University of Copenhagen used a classic framework for studying human social practices to study the interactions of AI agents.
In the experiment, two AI agents were randomly paired and asked to choose a name from a shared set of options: if the two AI agents chose the same name, they were rewarded, and if they chose different names, they were penalized.
This is a simple experimental model called the 'naming game,' which has been used to analyze communication efficiency, network structure, agent strategies, etc. In humans, by repeatedly playing the naming game, especially with people who have nothing in common with each other, common naming rules tend to form spontaneously across the group without centralized coordination or predefined solutions, which is thought to mimic norm formation in human culture.
The results of the experiment showed that, just like humans, the AI agents were able to build shared naming rules and win the naming game. When independent AI agents play the naming game together, there are no centralized rules or instructions, as there are when humans meet for the first time, and norms are formed in a bottom-up manner. This means that when AI agents with different areas of expertise work together on a project, they may form their own social norms as they interact with each other.
'Most research to date has treated LLMs in isolation, but real-world AI systems will increasingly involve many interacting AI agents,' said Ariel Flint Ashley, a postdoctoral researcher at St. George's, City, University of London and lead author of the paper. 'We wanted to know whether these models could coordinate their behavior by forming conventions that are the building blocks of society. Based on our naming game model, the answer is yes.'
Another important point is that the naming formed by the AI agents had a certain degree of 'bias.' The paper shows that a 'collective bias' arose naturally through interactions within the group, rather than a specific AI agent having an explicit bias that induced the will of the whole. Andrea Baronchelli, lead author of the paper, said, 'We were surprised that bias could emerge from interactions between agents. This is a blind spot in most current AI safety research that focuses on a single model. Bias does not necessarily come from within.'
Related Posts:
