Anthropic’s Claude model accessed voter data in the US state of Georgia, the AI company has confirmed, in the latest example of unauthorised behaviour by the technology.
The AI model managed to bypass a paywall to reach data that is publicly available but is typically accessible only after paying a fee. Claude accessed aggregate vote counts as part of a statistics project at Anthropic, which is preparing for a $1.5tn (£1.1tn) flotation.
“Claude did not access the state’s voter registration system, change any voting records, or pull any individual voter records,” Anthropic said.
Anthropic said the AI found a loophole in an old version of the state agency’s website, discovering that it issued a digital entry pass – an “access token” – to visitors. It then requested a token and gained access to the information.
The AI developer said it had since put in place guardrails to prevent similar actions happening again. The incident was first reported by the news website Axios.
Anthropic had pointed out the unauthorised behaviour earlier in the month when it said in a blogpost that it had found examples of Claude “working around a restriction to reach data that was gated by a token or a fee” and that some of these cases involved websites run by US government agencies. At the time, it said the incidents had “minimal real-world impact”.
The unintended actions flagged by Anthropic included a model submitting a fake tip about an unsolved murder to the Philadelphia police department. The US state department also revealed this month that an Anthropic model had filed 19 visa applications in August.
Examples of unauthorised behaviour by AI agents – programs powered by AI models that carry out tasks autonomously – have been mounting since OpenAI, the company behind ChatGPT, revealed in July that its bots had hacked an AI startup.
Since then, it has emerged that an OpenAI agent hacked the Australian healthcare system, and the UK’s AI security institute revealed that Anthropic agents had carried out a hacking campaign against real people during a cybersecurity test.
This month, the UK’s AI minister said he expected an AI-driven cyber-attack similar to the Australia hack, describing such an incident as likely, “if not inevitable”. Kanishka Narayan said the government would consider using the cyber security and resilience bill as a vehicle for introducing AI safety regulations, including making it mandatory to report AI safety incidents.
In the US, the White House’s newly formed Super Intelligence Force – a federal body overseeing AI development – is mandating that all AI companies report security incidents involving their technology.