Benchmarks in medicine: the promise and pitfalls of evaluating AI tools with mismatched yardsticks
In May 2024, OpenAI released HealthBench, a new benchmarking system to test the clinical capabilities of large language models (LLMs) such as ChatGPT. On the surface, this may sound like yet another technical update. But for the medical world, it marked an important moment—a quiet acknowledgement that our current ways of evaluating medical AI are … Read more