Enhancing Malignancy Detection and Tumor Classification in Pathology Reports: A Comparative Evaluation of Large Language Models

Neururer, Sabrina B.; Taha, Hasan; Muehlboeck, Helmut; Hickmann, Christoph; Gscheidlinger, Patricia; Richter, Stefan; Danler, Martin; Hackl, Werner O.; Ueberegger, Marko; Schweitzer, Marco; Pfeifer, Bernhard

doi:10.3233/SHTI250200

Enhancing Malignancy Detection and Tumor Classification in Pathology Reports: A Comparative Evaluation of Large Language Models

Authors

Sabrina B. Neururer, Hasan Taha, Helmut Muehlboeck, Christoph Hickmann, Patricia Gscheidlinger, Stefan Richter, Martin Danler, Werner O. Hackl, Marko Ueberegger, Marco Schweitzer, Bernhard Pfeifer

Pages

270 - 275

DOI

10.3233/SHTI250200

Background:

Cancer registries require accurate and efficient documentation of malignancies, yet current manual methods are time-consuming and error-prone.

Objectives:

This study evaluates the effectiveness of large language models (LLMs) in classifying malignancies and detecting tumor types from pathology reports.

Methods:

Using a synthetic dataset of 227 reports, the performance of four LLMs and a score-based algorithm was compared against expert-labeled gold standards.

Results:

The LLMs, particularly GPT-4o and Llama3.3, demonstrated high sensitivity and specificity in both malignancy detection and tumor classification, significantly outperforming traditional algorithms.

Conclusion:

LLMs enhance the accuracy and efficiency of cancer data classification and hold promise for improving public health monitoring and clinical decision-making.