$ articles --topic=evals
Evals
48 AI articles about Evals.
How We Optimized Browser Use for Deep AgentsThe post compares seven browser toolsets for Deep Agents and explains why the authors chose Stagehand from Browserbase. It also describes how they tuned it using LangSmith traces and evals, and shares the lessons they took from the process.Observability · via LangChain · 1d ago1 min[AINews] Inkling 2 Small: 17A-594B trained in 68 hours on 12T tokens, beating Muse SparkThis roundup covers Thinking Machines' Inkling 2 Small, a 594B-total, 17B-active multimodal MoE, including its benchmark gains over the predecessor, Tinker pricing, and the promised Apache 2.0 weights release. It also explains how the model compares with peers on coding, arenas, and domain evals, and the alignment problems early checkpoints showed.Evals · via Latent Space · 1d ago7 minHow Snorkel runs realistic agent simulations with ModalLearn how Snorkel uses Modal to run realistic agent simulations at scale. The article covers how Modal handles inference for the open-weight models Snorkel evaluates, and how its GPU and CPU sandboxes support millions of simulations each month.Agents · via Modal · 1d ago1 minIs the cheap model good enough? How to test a new model in your agentLearn how to test whether a cheaper model like Qwen3.8 Flash, wrapped in a manager/worker loop, can replace a frontier model in your agent. The guide covers defining success criteria, tracing runs in Arize AX, building evals and datasets, and comparing cost, latency, and pass rate across experiments.Observability · via Arize · 2d ago9 min[AINews] Behind Latent Space: Kill My SaaS, Jev & AI in 2026 | AIEi ShanghaiThis issue of AINews covers claims that frontier LLMs can read and write neural network weights directly, including a Qwen edit that lifted MATH-500 scores and its skeptical critiques. It also surveys new model releases, independent evals, provable inference from Anthropic, and audits that found flaws in popular benchmarks.Evals · via Latent Space · 3d ago7 minQwen 3.8 27B on my laptop can interpret model weights, tooSimon Willison tries the no-tools weight-reading test from the Sparks of Weight-Space Programming paper, running it on Kimi K3.1 Pro and on Qwen 3.8 27B locally. Qwen correctly identifies Pac-Man, Pong, and Breakout from raw weights but fails on Freeway, and the author argues the wider hype is overstated.Evals · via Simon Willison · 4d ago7 minStateBench: Testing how language models track state changes over long documentsStateBench is a benchmark for measuring how well language models keep track of information that changes across long documents, using four difficulty levels. The authors show where current models falter and how targeted training helps recurrent and hybrid architectures handle longer sequences.Evals · via Ai2 · 4d ago1 minLLMs are not superhuman at reading and writing neural network weightsThe author argues that a recent paper showing GPT-6 Astra reading and editing neural network weights is not superhuman. He says tasks like identifying games from weights and fixing a Qwen model's looping are things a competent ML researcher could do, using simple heatmaps and steering-style weight edits, so the result should not drive major changes to AI research.Agents · via Sean Goedecke · 4d ago5 minCWE-bench v2: More coverage, quality, and transparencyEvals · via Hugging Face · 4d ago1 minBuilding AI for Lawyers - Jacob Lauritzen, CTO of LegoraJacob Lauritzen, CTO of Legora, explains how the legal AI company builds agents for lawyers, including how it runs evals without access to customer data. The conversation also covers which tasks still need stronger models, how legal engineers and forward deployed engineers divide the work, and how AI is changing the billable hour.Agents · via Latent Space · 5d ago9 minBuilding Legal Agents When You Can't See the Data - Jacob Lauritzen, CTO of LegoraA podcast conversation with Legora CTO Jacob Lauritzen on building and testing a legal AI agent when customer data is invisible to the company. Topics include simulated test environments, citation accuracy gains, Tabular Review, model choice, and how forward-deployed and legal engineers work together.Agents · via Latent Space · 5d ago4 minBuild an AI Code Reviewer with OpenRouterShows how to build a Python code reviewer on the OpenRouter Responses API that loads changed files into a Linux sandbox, runs checks to confirm each suspected bug, and returns validated patches instead of prose suggestions. Reviews take one to four minutes and cost about $0.015 to $0.045 each.AI Coding Tools · via OpenRouter · 6d ago1 min