Abstract
Automated ICD coding is a multi-label classification task that involves assigning ICD codes to long clinical texts such as discharge summaries. It is an active research field and numerous studies demonstrate how using Natural language processing (NLP) techniques simplify this task making it cost-effective and aiding in accurate classifications. In recent years, transformers have extensively been used in NLP tasks, and offer a solution for accurately processing long texts and aid in automating ICD code classification. Transformers employ their self-attention mechanism to capture long-range dependencies. To enhance the state-of-the-art approaches for automated ICD-9 coding, we propose 2 models Long-LAT and Long-HiLAT leveraging the Longformer’s capability to process a large number of tokens. In Long-LAT, we introduce Label wise attention with the Clinical pretrained Longformer instead of solely relying on the classification head provided by the Longformer. This allows us to refine the classification process further by directing attention to the specific ICD codes. In Long-HiLAT, we make use of the Longformer’s sliding window attention to enhance the existing Hierarchical Label-wise Attention Transformer (HiLAT). This integration allows a more comprehensive analysis of the text and improves the model’s performance. Extensive experiments using the MIMIC-III top-50 and top-5 ICD code datasets show that Long-HiLAT achieved superior performance compared to the baseline models.