Anthropic publishes a report summarizing cases of chat AI abuse and countermeasures, reporting a service that uses AI to incite political ideology on more than 100 SNS accounts
AI company Anthropic has published a report outlining cases of chat AI abuse and countermeasures, reporting that it has thwarted coordinated efforts to spread specific political messages by creating multiple fake accounts on social media.
Detecting and Countering Malicious Uses of Claude \ Anthropic
Anthropic reports that the group behind the activities in question was a 'for-profit organization providing public opinion guidance services' that worked for multiple clients, each with different political and ideological affiliations, but that the organization ran all of its projects under the same system.
The investigation revealed that the group had created over 100 accounts on X (formerly Twitter) and Facebook, creating a network of politically aligned accounts. All of these accounts were fake, and they were all driven by AI such as Claude, with no human involvement, and they were all written in a variety of languages to promote certain ideological ideas.
These fake accounts wielded considerable influence by engaging with tens of thousands of 'real human' users, pushing politically biased narratives in communities across Europe, Iran, the United Arab Emirates and Kenya.
Anthropic said the cases uncovered in its investigation 'represent the evolution of AI-powered opinion-management services,' noting that 'the technical infrastructure is decoupled from political objectives; a single operator can serve multiple clients simultaneously; strategic and tactical decisions are made by the AI; and the content appears legitimate and patterns mimic human behavior, making it increasingly difficult to detect.'
Anthropic also reported that other cases included 'using Claude to rewrite open source toolkits to develop tools to scrape passwords and usernames related to security cameras,' 'a job scam that uses AI to communicate in foreign languages,' and 'novices using Claude to develop malware.'
Anthropic expects that as the barrier to adopting AI becomes lower, public opinion manipulation services like this one will become more commonplace, and has vowed to continue working to identify and block such AI-based influencer marketing activities, as well as share its findings with the broader security and safety community.
Related Posts:
