Large language models are powerful electronic health record encoders
TL;DR
General-purpose large language models can encode electronic health records converted into plain text to predict clinical outcomes just as accurately as specialized medical models, while avoiding the need for site-specific medical code vocabularies.
Problem / question
Machine learning on electronic health records (EHRs) is difficult due to complex, irregular data and site-specific medical codes, which limits the interoperability of specialized EHR foundation models that rely on fixed vocabularies and private training data.
Methods
The researchers converted structured EHR data into plain-text lists of medical code descriptions, capped at 8,192 tokens. They used general-purpose LLM embedding models, primarily Qwen3-Embedding-8B, to generate patient embeddings, which were then fed into logistic regression classifiers. They evaluated this pipeline on 15 clinical prediction tasks from the EHRSHOT benchmark (6,739 patients) and 25 tasks from the UK Biobank (387,464 patients), comparing it against the specialized EHR model CLMBR-T-Base, BioClinicalBERT, and count-based gradient boosted machines.
Key findings
On the EHRSHOT benchmark, Qwen3-Emb-8B matched the specialized CLMBR-T-Base model, both achieving a macro-AUROC of 0.769. On the UK Biobank dataset, Qwen3-Emb-8B slightly outperformed CLMBR-T-Base (macro-AUROC 0.751 versus 0.736), largely because CLMBR-T-Base could only map 16 percent (7,969 of 50,702) of the UK Biobank's medical codes, whereas the text-based LLM processed all codes. However, the LLM approach was much more computationally expensive, taking nearly 22 hours to encode the EHRSHOT dataset compared to just 6 minutes for CLMBR-T-Base.
Why it matters
This demonstrates that off-the-shelf LLMs can bypass the need for institution-specific code mapping and private medical pretraining data, offering a highly portable and flexible tool for clinical prediction tasks across different healthcare systems.
Limitations
The approach relies on a manually designed text serialization that only captures the most recent occurrence of each medical code, is constrained by context windows of up to 8,192 tokens, and incurs substantially higher computational and memory costs than specialized models.
Takeaway
Converting EHRs into plain text allows general-purpose LLMs to achieve state-of-the-art clinical prediction performance without requiring specialized medical vocabularies, though at a significant computational cost.