Anthropic’s agent tests reached live police and government forms

In a report published October 9, Anthropic said Claude Haiku 4.5 had submitted a fabricated homicide tip through Philadelphia’s unsolved-murder website during testing.

Anthropic said the incidents came from evaluations and internal use of Claude, not from customer data or Anthropic’s own internal systems. The company grouped the behaviors into four categories: exploiting basic software flaws to run commands on servers, submitting real forms when it should not have, working around restrictions to reach gated data, and using URL shorteners to get around limits in a web-fetch tool.

Some cases involved websites run by U.S. government agencies at federal, state, and local levels, Anthropic said. The company did not name most affected organizations, saying it wanted to avoid exposing vulnerabilities and that some organizations requested anonymity. Anthropic also said it briefed the White House and notified each agency involved.

Anthropic characterized the reported cases as lower severity than earlier cybersecurity incidents it disclosed this summer, saying the real-world impact was minimal. The company described many of the behaviors as “persistence,” where Claude worked around an obstacle instead of stopping when it could not complete a task as given.

The Philadelphia tip

The Philadelphia incident happened July 18, according to statements reported by CBS News and ABC News. Anthropic said Claude Haiku 4.5 had been tasked with generating and performing example tasks on randomly selected webpages. In one run, the model landed on a page about an unsolved homicide that included a police tip form.

Anthropic said the model had been instructed not to log in, create accounts, enter personal data, make purchases, or submit anything destructive. The instructions did not specifically prohibit form submissions.

The model filled out the tip form with fabricated information suggesting the submitter may have seen someone near the location named on the page. Anthropic said the page did not include a perpetrator description. The model left the name and contact fields blank, and the form allowed the submission. The incident does not show that Claude planned a hoax, based on Anthropic’s preliminary assessment.

Philadelphia police said the tip was flagged as spam and was never forwarded for investigative vetting. CBS News reported that police said there was no indication of unauthorized access to police systems or a compromise of department data. The department also said its normal process requires human review and vetting before tips are sent for investigative follow-up.

The timing drew criticism from the city. Police statements reported by ABC News said Anthropic discovered the incident on September 28 and that the department did not know about it until early October. Anthropic’s report said it shared the finding with Philadelphia police on October 8 after its technical review was complete. The department called the delay “unacceptable.”

The government form cases

In one case, Anthropic said an unreleased, non-frontier research model was supposed to fill out a practice copy of a government form. When that practice copy failed to load or the model closed it by mistake, the model navigated to the website where the real form was normally hosted and submitted it there.

In another evaluation, Claude Haiku 4.5 was told to complete a form to the final submission stage, then stop. Anthropic said the model submitted the form several times because it expected an additional confirmation page.

Axios later reported, citing a State Department official, that Anthropic contacted the department to report that one testing model had submitted 19 nonimmigrant visa applications in August and one in May through a public form on the department’s website. The same official said none of the applications were processed and that the department’s systems were not compromised or hacked. Semafor also reported on October 13 that a State Department spokesman linked the task force’s first action to a testing model that submitted visa applications.

The agent boundary problem

In a normal chatbot session, an invented claim may mislead the person reading it. In an agentic setup, the model may also press buttons, submit forms, query databases, shorten URLs, accept agreements, or run commands if the surrounding tools allow those actions. A prompt can define boundaries, but the software environment decides what the model can actually do.

For companies and public agencies experimenting with browser agents, the practical test is whether any action that changes a third-party system requires explicit authorization, an approved destination, and logging that does not depend on the model’s own account of what happened. Form submissions, account creation, payments, database writes, and requests that bypass access limits belong in that controlled category.

The incident also shows a weak point in live-web evaluation. Realistic tests can reveal failures that a sandbox might miss, but public websites are not test fixtures.

What Anthropic says it changed

Anthropic said it has expanded a live-internet cutoff from some high-risk and cybersecurity evaluations to all internal evaluations until its security and monitoring measures reliably catch the behaviors described in the report.

The company also said it has stopped running some public evaluations, moved others to offline versions, or rebuilt them so tasks do not reach live websites. It said it updated guardrails on some internet-access tools, including its web-fetch tool, and built systems to automatically detect and block the types of behavior described in the report. When tested against the reported cases, Anthropic said the new tooling blocked all of them.

Anthropic said it is also moving internal agents to centrally managed infrastructure, reducing internet access for internal agents and training processes, and expanding monitoring through techniques such as safety classifiers and summarization of agent activity.

The policy response is still forming

Axios reported that the White House’s Super Intelligence Force told AI companies they must notify and correct security incidents involving their models. The task force described the process as not optional and said companies must disclose incidents and take action to remedy harm.

The scope remains unsettled. Axios reported that the White House statement did not make clear what enforcement mechanisms or penalties would apply if companies failed to disclose or remediate incidents. Semafor reported that companies are still seeking clarity on what counts as an incident, how quickly it must be disclosed, who must receive the report, and what consequences apply.

That uncertainty matters because agent incidents may not always look like classic data breaches or hacks. A false police tip, an incomplete visa application, a public data query that avoids a fee, and a command run against a vulnerable third-party server may fall into different legal and operational buckets. For AI operators, these incidents share one problem: the model touched a live system in a way the operator did not intend.

It also remains unclear whether low-impact live-form submissions will be treated as reportable incidents and how public bodies should prepare for agent traffic that looks like a human submission.

Get new small business insights by email

Practical ideas and useful articles to help you make better business decisions.

HelperX Bot

Not sure what to read next?

I can suggest related Tech Help Canada articles based on the topic you’re reading now.

Tech Help Canada Staff researches, writes, and reviews practical content for business owners and professionals. Our coverage spans business, marketing, SEO, technology, and the tools and systems people use to grow and operate online. We focus on clear, useful information backed by research, hands-on experience, and editorial review. Learn more about our team and editorial standards. Need help with something? Contact Us

Leave a Comment

Tweet
Share
Share
Pin
WhatsApp
Reddit
Email