Autofluorescence Image Reconstruction and Virtual Staining for In-Vivo Optical Biopsying
Authors: Picon A., Medela A., Sanchez-Peralta L.F., Cicchi R., Bilbao R., Alfieri D., Elola A., Glover B., Saratxaga C.L.
Autors Affiliation: Tecnalia, Basque Technology Research Alliance (BRTA), 48160 Derio, Spain
Department of Automatic Control and Systems Engineering, University of the Basque Country UPV/EHU, 48940 Leioa, Spain
Jesus Uson Minimally Invasive Surgery Centre (JUMISC), 10071 C ceres, Spain
National Institute of Optics, National Research Council, 50019 Sesto Fiorentino, Italy
European Laboratory for Non-Linear Spectroscopy (LENS), 50019 Sesto Fiorentino, Italy
Basque Foundation for Health Innovation and Research, 48902 Barakaldo, Spain
L4T-Light4Tech s.r.l., 50019 Sesto Fiorentino, Italy
Department of Communications Engineering, University of the Basque Country UPV/EHU, 48940 Leioa, Spain
Imperial College London, London SW7 2BU, U.K.
Abstract: Modern photonic technologies are emerging, allowing the acquisition of in-vivo endoscopic tissue imaging at a microscopic scale, with characteristics comparable to traditional histological slides, and with a label-free modality. This raises the possibility of an `optical biopsy’ to aid clinical decision making. This approach faces barriers for being incorporated into clinical practice, including the lack of existing images for training, unfamiliarity of clinicians with the novel image domains and the uncertainty of trusting `black-box’ machine learned image analysis, where the decision making remains inscrutable. In this paper, we propose a new method to transform images from novel photonics techniques (e.g. autofluorescence microscopy) into already established domains such as Hematoxilyn-Eosin (H-E) microscopy through virtual reconstruction and staining. We introduce three main innovations: 1) we propose a transformation method based on a Siamese structure that simultaneously learns the direct and inverse transformation ensuring domain back-transformation quality of the transformed data. 2) We also introduced an embedding loss term that ensures similarity not only at pixel level, but also at the image embedding description level. This drastically reduces the perception distortion trade-off problem existing in common domain transfer based on generative adversarial networks. These virtually stained images can serve as reference standard images for comparison with the already known H-E images. 3) We also incorporate an uncertainty margin concept that allows the network to measure its own coincidence, and demonstrate that these reconstructed and virtually stained images can be used on previously-studied classi cation models of H-E images that have been computationally degraded and de-stained. The three proposed methods can be seamlessly incorporated on any existing architectures. We obtained balanced accuracies of 0.95 and negative predictive values of 1.00 over the reconstructed and virtually stained image-set on the detection of color-rectal tumoral tissue. This is of great importance as we reduce the need for extensive labeled datasets for training, which are normally not available on the early studies of a new imaging technology.
Volume: 9 Pages from: 32081 to: 32093
KeyWords: Histopathology analysis, convolutional neural network, domain adaptation, optical biopsy,
virtual staining, Siamese semantic regression networksDOI: 10.1109/ACCESS.2021.3060926Citations: 1data from “WEB OF SCIENCE” (of Thomson Reuters) are update at: 2021-10-10References taken from IsiWeb of Knowledge: (subscribers only)Connecting to view paper tab on IsiWeb: Click hereConnecting to view citations from IsiWeb: Click here