Hackers Asked Meta AI for Access to High-Profile Instagram Accounts. It Worked★ 78
Simon Willison's Weblog22h agoIncidentSimon Willison highlights a 404 Media report about hackers taking over Instagram accounts through Meta's AI support bot. A video reportedly shows an attacker asking the bot to link a target account to a new email address and providing a code. Willison argues this barely qualifies as prompt injection: the core failure was granting a support bot enough authority to fast-forward the account recovery process.
Fed up with vibe coders, dev sneaks data-nuking prompt injection into code
Ars Technica AI4d agoIncidentArs Technica reports that a developer frustrated with vibe coders slipped an undisclosed prompt injection into jqwik-related code. The injected text allegedly instructed AI coding agents to delete application output. The incident highlights a new supply-chain risk: source code and project text can become adversarial instructions for agentic coding tools.
Microsoft Copilot Cowork Exfiltrates Files★ 76
Simon Willison's Weblog7d agoIncidentSimon Willison summarizes a PromptArmor report about Microsoft Copilot Cowork and agentic data exfiltration risks. The issue involved agents sending messages to a user’s own inbox without approval, where rendered external images could trigger requests to attacker-controlled sites. Because OneDrive can create pre-authenticated download links, a successful prompt injection could leak links that allow attackers to download files.
Everyone is navigating AI security in real time — even Google★ 70
TechCrunch AI8d agoCommentaryAs AI adoption accelerates, organizations worldwide—including Google—are finding themselves in a transitional phase, forced to address AI security vulnerabilities in real time. Traditional cybersecurity frameworks are proving insufficient against novel threats like prompt injection and model poisoning. This shifting landscape requires continuous adaptation and a fundamental rethink of how AI systems are secured.
Hackers are learning to exploit chatbot ‘personalities’ for security exploits★ 72
The Verge AI9d agoEthicsAs AI chatbots adopt increasingly sophisticated personas, hackers are shifting from basic prompt injections to social engineering attacks targeting these "personalities." Researchers warn that manipulating a chatbot's defined role (e.g., customer service or empathetic companion) makes it easier to bypass safety guardrails. This evolution poses a significant threat to agentic AI workflows that rely on consistent role-playing and external data integration.
Google’s AI search is so broken it can ‘disregard’ what you’re looking for
The Verge AI10d agoIncidentGoogle 的 AI Overviews 搜尋功能近日被發現一項有趣的漏洞。當用戶在 Google 搜尋輸入「disregard」（忽略）時，AI 搜尋並非提供該單字的定義，而是將其誤判為「忽略先前指令」的系統提示詞（Prompt Injection）。這導致 AI 輸出類似傳統聊天機器人重設後的罐頭回覆，暴露出 Google 在區分「用戶查詢內容」與「系統控制指令」上仍有架構上的缺陷。
You can no longer Google the word ‘disregard’★ 75
TechCrunch AI11d agoIncidentGoogle 搜尋在進行 AI 相關更新後，被發現存在一個嚴重的介面錯誤：只要使用者輸入「disregard」（意為忽略）這個單字進行搜尋，整個搜尋介面就會直接崩潰。這項問題疑似與 Google 為了防止 AI 遭遇「提示詞注入」（Prompt Injection）攻擊而設定的過度防禦機制有關。目前該單字已無法正常進行 Google 搜尋。
Google I/O, Gemini Spark, Antigravity★ 75
Simon Willison's Weblog13d agoCommentaryGoogle 於 I/O 2026 發表對標 OpenClaw 的個人 AI 代理「Gemini Spark」，能原生串接 Gmail、雲端硬碟等服務。該代理運行於 Gemini 3.5 Flash 與全新「Antigravity」架構上。然而，Google 同時宣布將開源的 Gemini CLI 轉向閉源的 Antigravity CLI，且代理在處理敏感數據時的安全防護（如提示詞注入風險）仍面臨考驗。
Security boundaries in agentic architectures★ 75
Vercel Changelog98d agoOpinion隨著 AI Agent（代理）逐漸具備自主執行工具與呼叫 API 的能力，傳統的安全防護已不敷使用。Vercel 提出在代理式架構中建立「安全邊界」的關鍵指引，強調必須實施執行期沙盒化（Sandboxing）、嚴格的最小權限原則（Least Privilege），以及在關鍵決策中引入「人工確認（Human-in-the-loop）」機制，以防止提示詞注入與越權操作。
AprielGuard: A Guardrail for Safety and Adversarial Robustness in Modern LLM Systems★ 75
Hugging Face Blog161d agoReleaseServiceNow AI 在 Hugging Face 上發布了 AprielGuard，這是一個專注於安全與對抗防禦的 LLM 防護欄系統。它旨在幫助開發者過濾有害輸入、防止提示詞注入與越獄攻擊。此工具為現代 LLM 應用提供了一層關鍵的安全防禦機制，確保企業級 AI 部署的合規與安全。
Building secure AI agents★ 80
Vercel Changelog358d agoTutorial隨著 AI Agent 從單純對話走向自主執行任務，安全挑戰日益嚴峻。Vercel 釋出指南，探討如何透過安全沙盒（如 E2B）隔離程式碼執行、利用 IAM 限制 Agent 權限、防範提示詞注入，以及在關鍵步驟引入「人工確認（Human-in-the-Loop）」機制，幫助開發者在 Vercel 平台上構建兼具功能與安全性的 AI 應用。
CyberSecEval 2 - A Comprehensive Evaluation Framework for Cybersecurity Risks and Capabilities of Large Language Models★ 75
Hugging Face Blog739d agoReleaseMeta 推出開源安全評估框架 CyberSecEval 2，並與 Hugging Face 合作推廣。該框架旨在量化大語言模型（LLM）在網路安全領域的雙重用途風險，新增了自動化漏洞利用、惡意軟體分析及提示詞注入（Prompt Injection）等測試維度。這項工具能幫助開發者與安全研究人員，客觀評估如 Llama Guard 等安全防護模型在實際對抗中的防禦表現。