Transformers for Interpretable Classification of Histopathological Images

Recent studies have shown how machine learning techniques can help and support the analysis of expert physicians’ digitalized biopsy images. In particular, these techniques can improve the process related to some repetitive tasks, provide information, and speed up physician decision-making. Classifiers based on neural networks, particularly those that use Vision Transformers, have proven to be state-of-the-art in several classification tasks. In this paper, we apply vision transformers to classify histological images provided in the BreakHis and GasHisSDB datasets, which collect images from breast and gastric biopsies, respectively. The transformer architecture allows classification performances close to the state-of-the-art, also providing an explainable mechanism based on attention maps that localize and highlight the image regions relevant to the classifier’s decision.