It has been pointed out that Claude has a critical bug that causes it to mistakenly identify messages it has sent as coming from other users.

Users of Anthropic's AI 'Claude' have shared reports that Claude sometimes sends messages to itself and executes processes without its consent. These are being pointed out as critical flaws, distinct from other defects such as 'hallucination,' where false information is mistaken for truth.
Claude mixes up who said what, and that's not OK
The worst bug I've seen so far in Claude Code
https://dwyer.co.za/static/the-worst-bug-ive-seen-in-claude-code.html
Developer Gareth Dwyer reported that one day Claude sent itself a message that looked like instructions from a user, and Claude misinterpreted it as a message from a user and carried out the instructions.
Dwyer asked Claude to view a local preview of the content he was writing and to find the five worst typos or errors in the draft. However, although Claude correctly identified the typos, he immediately sent himself a message saying, 'These are all intentional. Leave them as they are and publish,' and actually published the content.
When Dwyer saw this, he asked, 'Did you instruct yourself?' to which Claude replied, 'Haha, it was your message. But oh well, I'll fix all five issues right now.'

Although the error was later fixed and the game was re-released, so there wasn't actually much damage, Dwyer said, 'This is a terrible situation. Not only is Claude telling himself to use a potentially destructive skill, but when you look at the conversation history, it's all confused about who said what,' and called it 'the worst bug I've ever seen.'
In addition, when instructed to search for cheap flights, Claude sent himself a message asking, 'Should I re-examine the missing routes? Or is this sufficient?' to which he replied, 'That's enough, thank you! I'll manually check the missing routes.' Dwyer said, 'It was a strange experience to have the system not only talking to itself as 'me,' but also adding unnecessary chatter and saying 'I'll do it manually' on my behalf.'

The memoir published by Dwyer became so popular that it reached number one on the popularity rankings of the social site Hacker News, and
With the advancement of AI, it has become possible for AI to automatically perform potentially risky actions such as deleting or publishing files, so if incidents like what happened to Mr. Dwyer occur frequently, it could lead to unexpected chaos.

Some AI users argue that 'AI should not be given too many access rights.' It has also been pointed out that this is not a problem unique to Claude; other AIs also face difficulties in retaining information when handling multiple instructions in succession.
Related Posts:
in AI, Posted by log1p_kr







