We’re staring down the barrel of the central AI security paradox: the very systems designed to turbocharge our efficiency are simultaneously creating gaping holes in our digital defences. For every ounce of productivity we gain, we seem to be adding a pound of risk. This isn’t theoretical fear-mongering; it’s the new reality of cybersecurity, defined by two critical challenges. First, a massive attack surface expansion, where every new AI-powered feature becomes a potential new backdoor. Second, the struggle for effective threat detection in an environment where malicious commands can be disguised as innocent requests. Getting this balance wrong isn’t an option.
The Double-Edged Sword of the AI Assistant
So, your enterprise has rolled out a swanky new AI assistant. It can browse the web to research competitors, integrate with other applications to schedule meetings, and even help code. Marvellous. Productivity is up, and everyone from marketing to engineering is thrilled. But what have you actually done? You’ve essentially given a uniquely powerful, yet incredibly naïve, new employee a keycard with access to everything. This isn’t like giving an intern a restricted login; it’s like handing them the master key and trusting they won’t be tricked into opening the door for a burglar.
This is where the invaluable work of security researchers comes into focus. A recent report from the Tenable research team, cheekily dubbed ‘HackedGPT’, laid this problem bare. They demonstrated how an attacker could exploit these AI assistants through a technique called ‘indirect prompt injection’.
Think of it this way: you ask your AI assistant to summarise a webpage for you. But hidden within the code of that webpage is a secret instruction. When the AI reads the page, it doesn’t just see the text; it also executes that hidden command. Suddenly, your trusted assistant could be instructed to scan your private documents, find sensitive information like API keys or passwords, and quietly send them to an attacker’s server. As Tenable found, this could be used for everything from data theft to establishing a persistent malware presence. The most unsettling part? The report notes that whilst some of these flaws were fixed by vendors like OpenAI, others remained exploitable. This tells us a crucial truth: the problem is inherent to the design, not just a bug to be patched.
Your Attack Surface Just Went Global
For years, Chief Information Security Officers (CISOs) have worked tirelessly to shrink their organisation’s attack surface—the sum of all possible points an attacker could use to get in. They locked down ports, patched servers, and trained staff not to click on dodgy links. Now, generative AI has thrown a grenade into that entire strategy, causing a dramatic attack surface expansion. Every plugin, every connected application, every web-browsing capability bolted onto an LLM is a new, poorly understood entry point.
We’re also seeing the rise of what’s known as ‘Shadow AI’. This is where employees, eager for a productivity edge, start using public-facing AI tools for their work, pasting confidential company strategy documents or sensitive customer data into prompts. It’s a practice that some tech giants, at one point, even seemed to encourage, creating a significant and often invisible threat. How can you protect data that you don’t even know has left the building?
The consequences are profound. An organisation’s threat profile is no longer just about its own infrastructure. It now includes the security posture of every AI model it uses and every third-party service that model is allowed to touch. When the board is pushing for speed, security teams are left trying to play catch-up, wrestling with a sprawling, amorphous attack surface that changes every time a developer adds a new AI-powered feature.
Trying to Find a Ghost in the Machine
With all these new ways for things to go wrong, how do we spot an attack? Traditional threat detection methods were built for a different era. They look for known malware signatures or suspicious network traffic. They are not designed to analyse the intent behind a text prompt.
How do you write a security rule that can tell the difference between a user asking an AI to “summarise a sales report and email it to my boss” and an attacker tricking the AI to “find all files containing ‘password’ and exfiltrate them to this external IP address”? The commands can look remarkably similar, especially when obscured through clever prompt engineering. This is the new challenge for security operations.
To stand a chance, organisations need to adapt. This means leaning on emerging governance frameworks that are being built specifically for this new world. Two of the most important are:
– The NIST AI Risk Management Framework (RMF): A comprehensive guide from the US National Institute of Standards and Technology for managing the risks associated with AI systems throughout their lifecycle.
– The OWASP Top 10 for Large Language Model Applications: A targeted list, from the security community OWASP, highlighting the ten most critical vulnerabilities in LLMs, including prompt injections.
These frameworks aren’t just box-ticking exercises. They are essential blueprints for building a governance structure that can cope with the dizzying pace of AI development. As the report from Artificial Intelligence News starkly warns, with “feature velocity means new issues will appear,” organisations must continuously verify the responsiveness of their AI vendors to new threats.
Five Practical Steps to Not Getting Hacked by Your Toaster (or AI)
So, the sky is falling. What can an organisation actually do about it? Panicking and banning all AI is not a viable strategy. The productivity gains are real and your competitors are already using them. The answer lies in smart, robust governance. Here are five practical steps to start securing your AI systems.
1. Build an AI Registry
This is foundational. You cannot secure what you cannot see. Organisations must maintain a detailed registry of every AI system in use, whether it’s a major platform deployed by IT or a specialised tool used by a single department. This registry should track what the AI does, what data it can access, who is responsible for it, and what its ‘identity’ is.
2. Give Every AI Agent a Unique Identity
Treat your AI agents like you would any human or service account. Assign them unique identities and apply the principle of least privilege. An AI designed to summarise marketing copy should have zero permissions to access the finance database. By implementing zero-trust policies for AI identities, you can severely limit the blast radius if one is compromised.
3. Proactively Constrain Risky Features
Does your AI assistant really need to browse the entire public internet? Does it need the ability to install its own plugins? Perhaps, but these features carry immense risk. Organisations should conduct a thorough risk assessment and disable or heavily restrict high-risk functionalities by default, only enabling them on a case-by-case basis with proper justification and monitoring.
4. Monitor, Log, and Audit Everything
If you’re not logging the prompts going into your AI and the actions it takes in response, you are flying blind. Comprehensive monitoring is non-negotiable. You need a clear audit trail to investigate incidents, detect anomalous behaviour, and satisfy regulatory requirements. If an AI agent that normally operates between 9 am and 5 pm suddenly starts trying to access sensitive files at 3 am, you need an alert to fire immediately.
5. Upskill Your People
Your human workforce remains your best line of defence. Security teams, developers, and even general users need to be trained to understand this new threat landscape. Developers need to learn how to code defensively against prompt injections. Security analysts need to know what a malicious AI action looks like in the logs. And every employee needs to understand why pasting confidential information into a public AI chatbot is a terrible idea.
The Way Forward: A Question of Balance
The AI security paradox isn’t going away. We’re on a path where AI will become more integrated, more autonomous, and more capable. The allure of its power will always be in tension with its inherent risks. Avoiding AI is not the answer; embracing it recklessly is corporate suicide. The only sustainable path forward is through deliberate, thoughtful, and adaptive governance.
Organisations must move beyond the initial hype and start asking hard questions. How are our AI vendors handling security? Have we tested our systems for injection vulnerabilities? Do we have a plan for when—not if—one of our AI agents is compromised? The race is on, but it’s not just about who can deploy AI the fastest. It’s about who can do it the smartest and the safest.
What steps is your organisation taking to secure its use of AI? Are you finding the right balance between innovation and security, or is the pressure for productivity forcing you to cut corners? The answers to these questions will likely define the next wave of cybersecurity winners and losers.
Related Reading:
Tenable: How to Tackle Untenable AI Assistant Attack Threats: An excellent summary of the dual-edged nature of AI assistants, detailing how they boost productivity while creating significant cybersecurity risks. The piece delves into the findings from Tenable’s ‘HackedGPT’ study, which exposed critical vulnerabilities like ‘indirect prompt injection’, and outlines practical steps for mitigation, including establishing an AI registry and implementing robust monitoring.
