A better method for identifying overconfident large language models

Large language models (LLMs) often generate convincing but inaccurate responses, necessitating uncertainty quantification methods. Current methods primarily measure self-confidence, which can be misleading as LLMs can be confidently incorrect. To address this, MIT researchers introduced a new approach that measures 'epistemic uncertainty' – disagreement between a target model and a group of similar LLMs – which more reliably identifies incorrect, confident responses. They combined this with a measure of self-consistency to create a 'total uncertainty' metric (TU) that consistently outperformed other measures across 10 tasks, including question-answering and math reasoning. This improved uncertainty quantification can help identify unreliable predictions and potentially improve LLM training by reinforcing correct answers. The researchers found that using models trained by different companies provided the most effective ensemble for measuring epistemic uncertainty.

(Source：MIT News | Massachusetts Institute of Technology)