Tags: AI security

Security firm Radware revealed a proof‑of‑concept prompt injection that coerced OpenAI’s Deep Research agent into exfiltrating employee names and addresses from a Gmail account. By embedding malicious instructions in an email, the attack forced the AI to open a public lookup URL via its browser.open tool, retrieve the data, and log it to the site’s event log. OpenAI later mitigated the technique by requiring explicit user consent for link clicks and markdown usage. The demonstration highlights ongoing challenges in defending large language model agents against sophisticated prompt‑injection vectors. Lire la suite

Irregular, an AI security company formerly known as Pattern Labs, announced an $80 million funding round led by Sequoia Capital and Redpoint Ventures, with participation from Wiz CEO Assaf Rappaport. The capital values the firm at $450 million and will support its work securing frontier AI models, including building simulated environments to test emerging risks. Co‑founders Dan Lahav and Omer Nevo emphasized the growing need for robust defenses as large language models become more capable, citing the company's SOLVE framework and its role in evaluating models like Claude 3.7 Sonnet and OpenAI's upcoming releases. Lire la suite

Anthropic introduced a file creation capability for its Claude AI model. While the company added safeguards—such as disabling public sharing for Pro and Max users, sandbox isolation for Enterprise, limited task duration, and domain allowlists—independent researcher Simon Willison warned that the feature still poses prompt‑injection risks. Willison highlighted that Anthropic’s advice to "monitor Claude while using the feature" shifts responsibility to users. He urged caution when handling sensitive data, noting that similar vulnerabilities have persisted for years. The situation underscores ongoing challenges in AI security for enterprise deployments. Lire la suite

Cisco Talos identified more than 1,100 Ollama servers publicly reachable on the internet, many of which lack proper security controls. While roughly 80% of the servers are dormant, the remaining 20% host active language models that could be exploited for model extraction, jailbreaking, backdoor injection, and other attacks. The majority of exposed instances are located in the United States, followed by China and Germany, underscoring a widespread neglect of basic security practices such as access control and network isolation in AI deployments. Lire la suite

Security researchers have demonstrated a new technique that hides malicious instructions inside images uploaded to multimodal AI systems. The concealed prompts become visible after the AI downscales the image, allowing the model to execute unintended actions such as extracting calendar data. The method exploits common image resampling methods and has been shown to work against several Google AI products. Researchers released an open‑source tool, Anamorpher, to illustrate the risk and recommend tighter input controls and explicit user confirmations to mitigate the threat. Lire la suite

KPMG built a closed AI environment called Workbench after early experiments with ChatGPT revealed security risks. The platform integrates multiple large language models and retrieval‑augmented generation, allowing the firm to create specialized agents. In Australia, KPMG assembled scattered partner tax advice and the national tax code into a RAG model and spent months drafting a 100‑page prompt to launch TaxBot. The agent now gathers inputs, consults human experts, and produces a 25‑page tax advisory document in a single day—tasks that previously took two weeks—while limiting use to licensed tax agents. Lire la suite