Vision Transformer-Based Pneumonia Detection for Chest X-Ray Images
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Pneumonia remains a leading global cause of mortality, necessitating timely and accurate diagnosis to prevent severeoutcomes. While conventional convolutional neural networks (CNNs) are effective in analyzing chest X-rays (CXRs), they often failto capture the global contextual information critical for detecting subtle pneumonia features. This study proposes a Vision Transformer(ViT)-based framework optimized for pneumonia diagnosis. Leveraging self-attention mechanisms, the model captures longrangespatial dependencies in CXRs. To address class imbalance, the architecture incorporates oversampling and weighted loss functions. Training was conducted on a publicly available Kaggle dataset comprising 5,863 images (4,273 pneumonia and 1,583 normal). Key enhancements include learnable positional encodings, patch-based input (16 × 16 patches), and linear learning rate scheduling(initial rate: 5e-5; warmup ratio: 0.1) to ensure stable training. The ViT model outperformed CNN baselines, achieving 97.61% accuracy,95% sensitivity, 98% specificity, and an AUC-ROC of 0.96, surpassing visual geometry group 16-layer network (92.14%) andresidual network with 50 layers (82.37%). Attention maps provided interpretable visualizations, highlighting pathological regions(e.g., lung opacities) consistent with clinical findings. Although computationally intensive, the framework demonstrated scalabilityand stability across 15 training epochs using gradient accumulation (steps = 4) to emulate larger batch sizes. These results underscorethe ViT's potential as a robust tool for early pneumonia detection, particularly in resource-constrained settings.
Abstract
Pneumonia remains a leading global cause of mortality, necessitating timely and accurate diagnosis to prevent severeoutcomes. While conventional convolutional neural networks (CNNs) are effective in analyzing chest X-rays (CXRs), they often failto capture the global contextual information critical for detecting subtle pneumonia features. This study proposes a Vision Transformer(ViT)-based framework optimized for pneumonia diagnosis. Leveraging self-attention mechanisms, the model captures longrangespatial dependencies in CXRs. To address class imbalance, the architecture incorporates oversampling and weighted loss functions. Training was conducted on a publicly available Kaggle dataset comprising 5,863 images (4,273 pneumonia and 1,583 normal). Key enhancements include learnable positional encodings, patch-based input (16 × 16 patches), and linear learning rate scheduling(initial rate: 5e-5; warmup ratio: 0.1) to ensure stable training. The ViT model outperformed CNN baselines, achieving 97.61% accuracy,95% sensitivity, 98% specificity, and an AUC-ROC of 0.96, surpassing visual geometry group 16-layer network (92.14%) andresidual network with 50 layers (82.37%). Attention maps provided interpretable visualizations, highlighting pathological regions(e.g., lung opacities) consistent with clinical findings. Although computationally intensive, the framework demonstrated scalabilityand stability across 15 training epochs using gradient accumulation (steps = 4) to emulate larger batch sizes. These results underscorethe ViT's potential as a robust tool for early pneumonia detection, particularly in resource-constrained settings.