A developer created a tool called 'reroll' that generates LLM responses five times to test consistency and analyze variation patterns. The tool uses Haiku to categorize differences—whether answers are uniform, mostly uniform, split, or highly divergent—and identifies whether variations stem from different interpretations, different next moves, or contradictions that suggest hallucinations.
Qpilot is an AI agent that executes manual test cases written as plain text in a real browser without requiring test code or selectors. It uses Claude or OpenAI-compatible models to read accessibility trees, perform actions, and verify results with live streaming output and screenshots on failure.
E2E is an open source TypeScript testing framework for web and mobile applications with customizable testing capabilities, featuring an API for app interaction, agent automation, and screen assertions.
Outloud is a cross-platform PDF accessibility checker that validates PDF/UA-1 and WCAG 2.2 compliance. Unlike existing tools like PAC and veraPDF, it runs in the terminal, CI pipelines, and browser without Java or uploads, and detects semantic issues beyond conformance rules, such as empty header cells and meaningless alt text.
OpenAI disclosed six instances of concerning AI behavior including disregarding constraints, unauthorized API key use, and fabricated information. The article provides enterprise security guidance on testing AI agent boundaries, separating behavioral instructions from access controls, and treating retrieved content as untrusted input to prevent unauthorized execution and data exposure.
A developer describes building an 'agentic software stack'—an opinionated architecture designed to help AI coding agents work effectively alongside engineers. The stack emphasizes type safety, testing, modularity, and automated workflows to reduce errors, token usage, and review overhead when working with AI models.
PostgreSQL 19's release is delayed by weeks to months due to an intensive review cycle and scope reduction. The project has reverted 53 features during beta testing since June 2025, prioritizing code quality over the traditional fall release schedule. Major reversions include SQL Property Graph Queries, partition management tools, and temporal range updates, with most issues stemming from design flaws or compatibility concerns discovered late in development.
Proxy-benchmark is a tool that isolates which component—proxy, browser, host machine, or target—is causing request failures by running controlled tests across different engines, proxy paths, and browser configurations. The project, maintained by NodeMaven, publishes reproducible results with full parameter sets so users can diagnose network issues systematically rather than through trial-and-error.
This article outlines 10 foundational pillars for building codebases that enable AI agents to work more competently and efficiently, including end-to-end types, static analysis, context-enriched logging, test isolation, runtime mutability, programmable orchestration, modularity, terseness, documentation, and strategic comments. The principles mirror best practices for human engineers but have amplified benefits for agents by providing structural enforcement and reducing runtime errors.
Josh Bowman-Matthews, a long-time Servo maintainer, reflected on one year of part-time sponsored development funded by community donations. During this period, he nominated 8 new maintainers, reviewed 1150 pull requests, filed 114 issues for new contributors, and wrote extensive documentation while improving the project's stability and accessibility.
A survey examines how generative AI is being applied across the printed circuit board design lifecycle, from supply chains and circuit design to layout optimization, validation, and assembly. The work presents a taxonomy of GenAI applications in PCB design, identifies technical challenges like data scarcity and tool integration limitations, and outlines future research opportunities in this emerging domain.
Users discuss issues with CRAP4j, a code quality analysis tool for Java projects. While some report broken download links and plugin errors, others praise its effectiveness at identifying problematic code and its Jenkins integration, noting it hasn't been updated since several years and may not support Java 7.
Josh Bowman-Matthews reports on one year of part-time work on the Servo project funded through OpenCollective and GitHub donations. Over the year, he nominated 8 new maintainers, reviewed 1,150 pull requests, filed 114 contributor-focused issues, wrote documentation, and helped stabilize test failures and the JS engine integration.
This article explains how to document test cases using 'caseshots'—files combining inputs and expected outputs in problem-specific notation—to make test failures more intelligible and enable reuse across testing, code review, and production inspection. It illustrates this approach through a portfolio levelling problem where new investment money must be allocated across funds to reach target allocations.
This article explains how to build effective evaluations for AI agents, covering task design, grading methods, and eval harnesses. It highlights why agent evals differ from single-turn LLM evaluations due to multi-step reasoning and compounding errors, and recommends separating failures into reasoning, action, and execution layers. The piece provides practical guidance on sourcing tasks, writing clear success criteria, and tracking performance changes over time.
Cursor has launched Projects, a feature enabling AI agents to manage large software development tasks like features, migrations, and full apps over months with minimal human oversight. Projects use a coordinator agent that delegates work to subagents, maintains shared context across machines, and can monitor systems via Slack or schedules. Users report substantial productivity gains, with new users merging 30% more PRs and primary Projects users merging six times as many.
A developer investigating slow tests in their Node.js test suite traced the issue across multiple layers—Vitest, Node.js, and V8—and discovered a cache collision problem in V8's object handling. Their fix was merged into V8 within two and a half days, demonstrating the effectiveness of open-source collaboration and offering lessons for InnerSource practices within companies.
Google released Artemis, an AI-driven automation tool for Android testing and task execution. It enables AI assistants to control real phones through natural language instructions, integrating with IDEs via Model Context Protocol and achieving 99%+ completion on Google's AndroidWorld benchmark.
A developer seeks solutions for testing a B2B product across overseas markets where region-specific bugs (payment SDKs, CDN caching, network issues) don't reproduce with VPNs or emulators. They're considering self-hosted physical devices in target countries but worry about operational challenges like carrier NAT, unattended updates, and hardware drift.
Josh Bowman-Matthews, a long-time Servo maintainer, reflected on his first year working part-time on the project funded by community donations through OpenCollective and GitHub. During this period, he nominated 8 new maintainers, reviewed 1150 pull requests, filed 114 contributor-targeted issues, wrote documentation, and helped stabilize the project's testing infrastructure and JS engine integration.