Abstract

Pre-trained large language models (LLMs) have become the leading approach in modeling medical language for Natural Language Processing (NLP) in clinical notes. This paper presents a systematical comparative analysis on LLMs-based clinical data analytics, where LLMs-based models include Bert Base, BioBert, two variations of Bio+Clinical Bert, RoBerta, and Clinical Longformer on three of tasks of information extraction on Electronic Health Records (EHRs) from Track 1 of Harvard Medical School’s 2022 National Clinical NLP Challenges (N2C2). Experimental results demonstrate that these pre-trained LLMs are effective in detecting medication and medication events, while Bert Base, pre-trained on general domain data showed to be the most effective for classifying the context of events related to medications.