Medical exam question answering (MedQA USMLE)

Overview

Four-option questions in the style of the US Medical Licensing Examination (MedQA USMLE, official test split): accuracy of a language model choosing the most likely answer letter with no training, and of a pairwise logistic probe on frozen embeddings of each question with each option.

Leaderboard

Ranked results, with their source, uncertainty and floors, the protocol and a sample of the data are on the Leaderboard tab.