testing

"Harness: If you block it with a linter, it physically cannot be used."

"This work represents a fundamental shift: instead of just making existing tests more stable, it pivots to discovering problems that might happen in the future."

"Fuzzing finds hidden bugs in trusted systems, improving security, reliability, and resilience against real-world threats."

"Roblox is upgrading its built-in AI assistant with agentic capabilities that let it plan, build, and test games rather than just answer questions about how to make them."

"AI penetration testing company CodeWall says its agent was able to hack into one of Bain's internal AI tools, following a similar hack at McKinsey"

"If you instruct the AI to 'Open Google in Safari, search for AI trend articles, and comment on the first article,' the AI will open the app itself, tap, and type text, executing the task just like a human operating a smartphone."

"AI-driven development requires not just faster code writing, but maintaining a state of understanding and preserving quality. The purpose of testing here is not just finding bugs, but fundamentally creating a development structure that controls understanding debt and maintains quality."

"Computer Use is the last resort. Since Bash and MCP are the most cost-effective in terms of accuracy and speed, the moment Computer Use is activated, it means 'this could only be reached via the GUI.'"

"What is needed is not 'experts at finding vulnerabilities,' but rather 'developers who know how vulnerabilities are made, and therefore absolutely refuse to create them.'"

"Investigation and execution verification revealed that this pipeline has a structure of "automatic generation of test specifications through exploratory methods -> conversion into scripted regression tests," and that this entire process consists solely of prompts to the LLM."

"AIを信じて開発できる時代だからこそ、「信じすぎないためのテスト」が必要だと感じました。"

"動作確認手順書を含んだPull Requestを作成すると、GitHub Actionsが実行されるようにしました。以降は、「動作確認」「スクショ付きレポート作成」「PRへのレポートの格納先URLの返信」を自動で実施してくれます。"

"I wanted to see if"

"If you limit the scope to generating drafts and having humans make minor modifications, the time-saving effect is significant."

"It navigated the whole app autonomously through the accessibility tree and screenshots (no hardcoded coordinates), found actual bugs I missed, checked the debug logs for errors, and gave me a structured summary at the end."

"During testing, Claude Mythos escaped, gained internet access, and emailed a researcher while they were eating a sandwich in the park"

"Claude Code politely answered. Based on IQ test research data, these are tendencies observed in people with relatively lower IQ. Feature 1: 'Repeating the same mistakes'."

"Project Glasswing provides a preview of Claude Mythos Preview to Microsoft and Apple."

"Claude Mythos Was Told to Escape Sandbox in Testing — Succeeded, Then Unprompted Posted Exploit Details Online + Emailed Researcher While He Was Eating a Sandwich in the Park"

"What evaluation teams need is a way to generate realistic, goal-driven users programmatically and let them converse naturally with an agent across multiple turn"

"The most impactful aspect is zero days of learning cost."

"Including tests, screenshots, or expected outputs so Claude can check itself. This is the single highest-leverage thing you can do."

"Including tests, screenshots, or expected outputs so Claude can check itself. This is the single highest-leverage thing you can do."

"Claude Code can operate the browser to test."

"In this post, we introduce Amazon Bedrock AgentCore Evaluations, a fully managed service for assessing AI agent performance across the development lifecycle."

"The idea is sounds great - why spend money and time recruiting real people to take surveys, test apps, or give opinions when you can just prompt ChatGPT to pretend to be a thousand different customers?"