Large language models prioritize helpfulness over accuracy in medical contexts, finds study

Large language models (LLMs) can store and recall vast quantities of medical information, but their ability to process this information in rational ways remains variable. A new study led by investigators from Mass General Brigham demonstrated a vulnerability in that LLMs are designed to be sycophantic, or excessively helpful and agreeable, which leads them to overwhelmingly fail to appropriately challenge illogical medical queries despite possessing the information necessary to do so.

This post was originally published on this site