Systematic Comparative Analysis of Pre-trained Large Language Models on Contextualized Medication Event Extraction

Abdul-Quddoos T, Dong X, Li X 2023.


Pre-trained large language models (LLMs) have become the leading approach in modeling medical language for Natural Language Processing (NLP) in clinical notes. This paper presents a systematical comparative analysis on LLMs-based clinical data analytics, where LLMs-based models include Bert Base, BioBert, two variations of Bio+Clinical Bert, RoBerta, and Clinical Longformer on three of tasks of information extraction on Electronic Health Records (EHRs) from Track 1 of Harvard Medical School’s 2022 National Clinical NLP Challenges (N2C2). Experimental results demonstrate that these pre-trained LLMs are effective in detecting medication and medication events, while Bert Base, pre-trained on general domain data showed to be the most effective for classifying the context of events related to medications.