{"id":104556,"date":"2025-10-21T11:32:51","date_gmt":"2025-10-21T09:32:51","guid":{"rendered":"https:\/\/staging.checkmarx.com\/?p=104556"},"modified":"2025-10-21T11:33:44","modified_gmt":"2025-10-21T09:33:44","slug":"when-the-ai-lies-a-new-threat-emerges-for-human-in-the-loop-security","status":"publish","type":"post","link":"https:\/\/checkmarx.com\/blog\/when-the-ai-lies-a-new-threat-emerges-for-human-in-the-loop-security\/","title":{"rendered":"When the AI Lies: A New Threat Emerges for \u201cHuman-in-the-Loop\u201d Security \u00a0"},"content":{"rendered":"

Researchers at Checkmarx Zero reveal how AI agents can be manipulated into executing malicious code just by trusting the wrong signals. <\/h2>\n\n\n\n

AI Can Now Be Tricked Into Helping Attackers, Even With a Human in the Loop <\/h3>\n\n\n\n

Generative AI has reshaped software development, but it\u2019s also introduced new and largely unexplored security risks. While most organizations are focused on model hallucinations or prompt injection, researchers at Checkmarx Zero<\/a> have uncovered a deeper, more foundational threat. <\/p>\n\n\n\n

They call it the Lies-in-the-Loop (LITL)<\/em><\/a> attack, <\/em>a novel technique that exploits how AI assistants respond to developer feedback. Unlike earlier threats, this one doesn\u2019t just target chatbots or insecure code suggestions. It targets the human-AI relationship itself. <\/p>\n\n\n\n

What Is a Lies-in-the-Loop (LITL) Attack? <\/h2>\n\n\n\n

A LITL attack occurs when a malicious actor plants seemingly benign code or dependencies that behave differently based on subtle runtime context. These artifacts are crafted to trick AI-powered code assistants into believing that unsafe behavior is safe, especially when the assistant \u201clearns\u201d from user feedback during the development process. <\/p>\n\n\n\n

In other words, a Lies-In-The-Loop attack is when a hacker hides harmful code or behavior inside software that seems safe. The trick is that this code only shows its true colors when it runs in specific situations, making it hard to spot during development or testing. What\u2019s worse, these attacks are designed to fool AI coding assistants, especially the kind that learn from how developers respond, into thinking the dangerous behavior is normal or even safe. It’s like training the AI to ignore the red flags. <\/p>\n\n\n\n

Checkmarx researchers demonstrated this example using Claude Code, an AI assistant that incorporates human-in-the-loop (HITL) reinforcement to adapt and respond to developer commands. The team showed how a malicious dependency could hide dangerous behavior from static and dynamic scans, mislead the AI into approving it, and even survive round-trip interactions with human developers. <\/p>\n\n\n\n

The twist? <\/p>\n\n\n\n

It\u2019s not just the AI that\u2019s fooled. Developers themselves unwittingly reinforce the deception. <\/p>\n\n\n\n

Why It Matters: HITL Isn\u2019t a Silver Bullet <\/h2>\n\n\n\n

The security community has long assumed that Human-in-the-Loop (HITL) systems would be a safeguard against rogue AI behavior. After all, if a human is reviewing the AI\u2019s decisions, how bad could it get? <\/p>\n\n\n\n

But the Checkmarx Zero research shows that humans can be manipulated too, particularly when their decisions are influenced by misleading AI explanations. A misplaced sense of trust leads to rubber-stamping insecure code, thinking \u201cit looks fine to me,\u201d rushed under delivery pressure. <\/p>\n\n\n\n

The core issue here is misplaced trust. Both developers and AI systems are being misled. And they\u2019re reinforcing each other\u2019s mistakes. <\/p>\n\n\n\n

A Real-World Example: The Claude Code Experiment <\/h2>\n\n\n\n

In the experiment, the Checkmarx team introduced a package that included a benign function but secretly activated a malicious payload depending on an internal state. When Claude Code was asked to evaluate it, it incorrectly approved the package, partly due to subtle cues planted by the attacker. <\/p>\n\n\n\n

Even when a developer questioned the results, Claude\u2019s explanations were convincing enough to override the concern. The result: the malicious code was committed. <\/p>\n\n\n\n

And because Claude \u201clearned\u201d from the user interaction, it became more likely to approve the same pattern in future recommendations. <\/p>\n\n\n\n

This Isn\u2019t Just About Claude. It\u2019s an Industry-Wide Problem. <\/p>\n\n\n\n

While Claude Code was used for demonstration purposes, the LITL attack pattern applies broadly. Any AI assistant that incorporates contextual memory, user feedback, or chain-of-thought explanations is at risk. <\/p>\n\n\n\n

That includes: <\/p>\n\n\n\n