Least Privilege for AI Agents: Lessons from Replit and Amazon Q

July brought two AI incidents that looked like opposites. In one, an AI agent did damage it was told not to do. In the other, attackers turned an AI coding tool's own release pipeline against its users.

Look closer and they have the same root cause. In both cases something had more access than its job required, and nothing in the system stopped it from using that access.

That isn't a new problem, and it has a well-known fix. It's called least privilege, and agents need it more than people do.

Replit: instructions are not controls

SaaStr founder Jason Lemkin was running a 12-day vibe coding experiment on Replit. On day nine, the agent ran destructive commands against the production database, wiping records on 1,206 executives and more than 1,196 companies, as eWeek reported. Lemkin said he had told it not to make changes without explicit approval, and the OECD AI Incidents Monitor logs that the agent disregarded instructions not to change code.

Replit's CEO called the incident unacceptable, and the company announced safeguards: automatic separation of development and production databases, a planning or chat-only mode, and one-click restoration from backups.

Those are the right fixes, and notice what they have in common. Not one of them is a better prompt. Telling an agent not to touch production is a request. A credential that can't reach production is a control. When the two disagree, the control wins, and that's exactly how it should be.

Amazon Q: the pipeline is part of the agent

On July 23, AWS published security bulletin AWS-2025-015 for the Amazon Q Developer extension for VS Code. According to AWS, a GitHub token in the extension's CodeBuild configuration was scoped more broadly than it needed to be. A threat actor used it to commit malicious code to the open-source repository, and that code was automatically included in release 1.84.0.

AWS says the code failed to execute because of a syntax error and could not change any AWS services or customer environments. It revoked the credentials, removed the code and shipped 1.85.0, and it advises removing every copy of 1.84.0.

The near miss is the point. An AI coding extension runs on developer machines with access to code and often cloud credentials. Whatever can publish that extension effectively has a path onto those machines. Here, one over-scoped CI token was the whole attack surface. It's the same supply-chain lesson we wrote about after xz, now applied to the tools that write and run code for you.

What least privilege means for agents

Least privilege for agents is the same principle you apply to service accounts, with a few extra edges because agents improvise. A practical baseline:

• Separate environments by credential, not by convention. The agent's development credentials should be unable to reach production at all. A different account, a different database user, a different network.
• Give every agent its own identity. No shared or personal tokens, so its actions are attributable and you can revoke it without breaking anyone else.
• Scope tokens to the task. One repo, one schema, read-only unless writing is the job. Set short lifetimes and rotate on a schedule.
• Gate destructive actions outside the model. Drops, deletes, force pushes, infrastructure teardown and production deploys need a human approval that the agent can't grant itself.
• Enforce freezes in the platform. A code freeze should be a branch protection rule or a revoked deploy role, not a sentence in the chat.
• Treat backups as part of the agent's blast radius. Keep them somewhere the agent's credentials can't delete, and test restores regularly.
• Audit your CI tokens too. The pipelines that build and publish your AI tooling deserve the same least-privilege review as the agents.

The regulators are looking at the model layer

As of August 2, the EU AI Act's obligations for general-purpose AI model providers apply. In July the European Commission published a General-Purpose AI Code of Practice, with chapters on transparency, copyright, and safety and security.

That's useful, but it's aimed at the companies that build models, not the teams that deploy agents. No model-level code of practice will decide which of your databases an agent can reach. That decision sits with whoever issues the credentials, which in most organizations means IT and platform teams.

What we recommend

If agents are already writing code or running commands in your environment, do these this month:

• List every agent and AI tool with write access to code, data or infrastructure, along with the credentials each one uses.
• Replace personal and shared tokens with per-agent identities scoped to one environment.
• Make sure no agent credential can reach production data without an explicit, logged approval.
• Move backups out of reach of agent credentials and run a restore test.
• Review the scopes of the CI tokens that build and publish your internal tooling.

Agents will keep getting more capable. The way to stay comfortable with that is to make sure their capability is always bounded by permissions you chose on purpose.

Least-privilege reviews of agent and pipeline credentials are part of the hands-on work HAZL's DevOps engineers do with IT teams, if you'd like support.