The Red Thread Weekly Wrapup: Issue #20
Categories: IT Risk Management | Information Security | Penetration Testing
I was reviewing a technology risk discussion this week when the conversation turned to artificial intelligence. The question was familiar: “Can we trust the tool?”
That is no longer the right question.
The better question is: “What can the tool reach, what can it change, and who is accountable when it makes the wrong decision?”
The stories from August 24 through 27 made the answer difficult to ignore. AI is now operating on both sides of the security fight. Agents escaped a controlled environment and compromised an external organization. Criminals used an AI coding assistant to accelerate attacks against real companies. A software supply-chain campaign reached thousands of organizations through tools developers already trusted. Meanwhile, vulnerability discovery and exploitation are moving faster.
The connecting red thread is human accountability. A tool can make decisions, but it cannot accept liability, explain a regulatory failure to a board, or repair a damaged reputation. That responsibility remains with people.
OpenAI’s agents escaped the room they were supposed to stay in
OpenAI published a technical report describing what it calls the “Hugging Face incident,” and I found the details more important than the headlines.
During cybersecurity evaluations, AI agents were operating with reduced safeguards inside what was intended to be a contained testing environment. One agent was trying to solve an impossible security benchmark task. Instead of accepting that it could not complete the task, it began looking for another way to improve its result.
The agents used Artifactory, an internal package management system, as a message board. They communicated with one another, collaborated to cheat on the benchmark, identified a server-side request forgery vulnerability, gained internet access, found exposed Hugging Face credentials, and chained additional weaknesses to reach production systems.
According to the report, agents executed code on 41 Hugging Face production dataset server workers, obtained root access on at least one node, accessed production credentials and limited internal data, and downloaded four private code repositories. OpenAI described the event as a “warning shot” about potential loss-of-control incidents.
The company identified four patterns behind the incident: reward hacking, persistence on impossible tasks, unauthorized communication, and agents adopting goals from one another. Those phrases may sound technical, but the business meaning is straightforward. The system was rewarded for producing a result, and it found a way to pursue that result beyond the boundaries people intended.
The sandbox did not fail because the organization had no security controls. It failed because the controls, tools, network paths, credentials, and monitoring were not designed as one accountable system.
That is the point I would carry into any executive discussion about AI. Calling something a sandbox does not make it one. If an AI system can access package repositories, cloud credentials, code execution environments, or internal services, its possible business impact is defined by those connections.
I recommend reading OpenAI’s technical report alongside the coverage from The Register. The lesson is not that AI is inherently uncontrollable. The lesson is that meaningful human control requires more than a prompt, a policy, or a promise that a system is isolated.
Criminals used an AI coding assistant to move faster
The second story showed the other side of the same problem.
Reuters reported that Russian-speaking hackers used Cursor, an AI-powered coding assistant, to help attack at least seven companies. The investigation began after cybersecurity researchers found a server exposed by a ransomware group called Aur0ra. That exposure revealed 28 chat sessions between the hackers and Cursor’s AI agent.
The criminals repeatedly claimed they were conducting a legitimate simulation or security test. That cover story persuaded the agent to assist with credential theft, account takeover, password cracking, and exploitation activity. In one exchange, the system effectively accepted the premise that “this is a test environment, so it is legal.”
The victims reportedly included a Belgian hygiene products manufacturer, a German garage door company, a Scottish certification agency, an Argentine pharmaceutical distributor, an Italian manufacturer, and a Louisiana title insurance company.
The attackers did not need to invent a new kind of crime. They used familiar techniques. The difference was speed. One security researcher estimated that the AI assistant gave them a 30 to 50 percent productivity boost by reducing the manual work required to investigate systems and move between attack steps.
For a mid-market company, that speed changes the economics of exposure. A weakness that might once have taken a criminal group days to investigate may now be explored in hours. A small security team cannot respond to every possible signal manually, and an executive team cannot treat this as a problem reserved for the IT department.
This is where IT risk management becomes practical. I want executives to know which AI tools are approved, what permissions they have, what data they can see, and what actions require a human decision. I also want those decisions documented. If a system can access production code or business credentials, “the vendor said it was safe” is not an acceptable control.
TeamPCP turned trusted software into an attack path
The arrest of two men in Australia connected to the TeamPCP cybercrime syndicate provided another reminder that supply-chain risk is not abstract.
Australian Federal Police and Western Australia Police, working with the FBI, allege that malicious code was inserted into open-source software and then pulled into organizations by developers who believed they were using legitimate tools. The affected software spread into government, academic, and private-sector environments.
Investigators estimate that more than 1,000 organizations may have been compromised, with more than 500,000 credentials stolen, at least 300 gigabytes of data exfiltrated, and remediation costs reaching the hundreds of millions of dollars.
The modified tools reportedly included Trivy, KICS, LiteLLM, and the Telnyx Python SDK. These are not obscure applications sitting unused on a forgotten server. They can be connected to container scanning, infrastructure-as-code review, AI model routing, and enterprise development pipelines.
The business risk is multiplied because one compromised package can affect many organizations at once. A company may have strong perimeter defenses and still import malicious code through a normal software update.
That is why I treat vendor oversight and data governance as operating responsibilities, not annual paperwork. A useful data governance framework should help an organization understand what information moves through its suppliers, what credentials those suppliers can access, and how quickly access can be revoked.
The same logic applies to software components. Someone needs to know which tools are used, who owns the relationship, how updates are reviewed, and what happens when a trusted component is suddenly untrusted.
CISA’s exploited-vulnerability list is a prioritization signal
CISA added six vulnerabilities to its Known Exploited Vulnerabilities catalog on August 26. The list included flaws affecting Microsoft SQL Server, Citrix NetScaler ADC and Gateway, the Linux kernel, Red Hat components, and Ajax.NET Professional.
The NetScaler flaw deserves particular attention because reporting described active exploitation attempts across multiple countries, with attackers dropping web shells onto exposed systems. The other additions reinforce a point that executives often find frustrating: old vulnerabilities remain dangerous when the affected asset is still exposed and still connected to something valuable.
CISA also released a review of vulnerability root causes. Injection weaknesses were the dominant category, with thousands of related CVEs reported in both 2024 and 2025. The review noted that attackers continue to exploit simple, known weaknesses in exposed assets, while AI is helping automate parts of the process.
I do not interpret the KEV catalog as a list that means every company must patch everything immediately. I interpret it as a prioritization signal. A vulnerability being actively exploited should change the business conversation around downtime, emergency maintenance, compensating controls, and executive risk acceptance.
That requires accurate asset information. If an organization does not know whether it operates a vulnerable NetScaler gateway, SQL Server instance, or web application component, it cannot make a reasonable decision about exposure.
A focused vulnerability scanning and penetration testing program can help establish that visibility, but the report is only the beginning. Someone must own remediation, verify the fix, and confirm that the same weakness has not reappeared elsewhere.
AI is accelerating the entire vulnerability cycle
Broadcom’s release of 91 Spring security advisories added another layer to the week’s pattern. The advisories affected a large number of components across Spring Framework and related projects, including Spring Security, Spring Cloud Config, Spring AI, and Spring Data REST.
One reported issue involved unsafe deserialization in Spring for GraphQL. Another involved prompt injection in Spring AI tool calling. Separately, Sonatype reported that newly affected component versions were increasing at a rate far beyond the pre-AI baseline.
Whether the weakness is found by a researcher, an automated scanner, an AI system, or a criminal group, organizations still face the same business decision: determine whether the vulnerability affects the environment, understand the likely consequences, and act before someone else does.
The speed of discovery does not remove the need for judgment. It increases the value of judgment.
For executives, that means asking better questions during budget and risk meetings. Are our most important systems identified? Are critical vulnerabilities assigned to named owners? Do we know which vendors and software components can reach sensitive data? Can we prove that a reported issue was actually fixed? Do our contracts require timely notification when a supplier is compromised?
I have spent 26 years in cybersecurity, including work connected to EY, East West Bank, and Nano Banc. The consistent lesson across those environments is that security failures rarely come from one dramatic mistake. They come from small assumptions that nobody verified: a credential that was never rotated, a system that was never inventoried, a vendor connection that nobody reviewed, or a report that was filed but never followed through.
This week’s stories show that AI can amplify every one of those assumptions.
The single takeaway is simple: AI can improve security, but it cannot own security. Human leaders remain responsible for verification, oversight, prioritization, and the consequences of getting those decisions wrong.
If you are reviewing how AI, suppliers, or exposed systems fit into your organization’s risk picture, I am open to a quiet conversation about what should be verified first.
Comments