2 weeks ago · ai

Artificial Analysis overhauls its AI Intelligence Index, replacing popular benchmarks with 'real-world' tests

The arms race to build smarter AI models has a measurement problem: the tests used to rank them are becoming obsolete almost as quickly as the models improve. O...

1 month ago · ai

Measuring AI Ability to Complete Long Tasks

Article URL: https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ Comments URL: https://news.ycombinator.com/item?id=46342166 Points: 1...

1 month ago · ai

Binary weighted evaluations...how to

1. What is a binary weighted evaluation? At a high level: - Define a set of binary criteria for a task. Each criterion is a question that can be answered with...