Autofluorescence Image Reconstruction and Virtual Staining for In-Vivo Optical Biopsying

Year: 2021

Authors: Picon A., Medela A., Sanchez-Peralta L.F., Cicchi R., Bilbao R., Alfieri D., Elola A., Glover B., Saratxaga C.L.

Autors Affiliation: Tecnalia Basque Technol Res Alliance BRTA, Derio 48160, Spain; Univ Basque Country UPV EHU, Dept Automat Control & Syst Engn, Leioa 48940, Spain; Jesus Uson Minimally Invas Surg Ctr JUMISC, Caceres 10071, Spain; CNR, Natl Inst Opt, I-50019 Sesto Fiorentino, Italy; European Lab Nonlinear Spect LENS, I-50019 Sesto Fiorentino, Italy; Basque Fdn Hlth Innovat & Res, Baracaldo 48902, Spain; L4T Light4Tech Srl, I-50019 Sesto Fiorentino, Italy; Univ Basque Country UPV EHU, Dept Commun Engn, Leioa 48940, Spain; Imperial Coll London, London SW7 2BU, England.

Abstract: Modern photonic technologies are emerging, allowing the acquisition of in-vivo endoscopic tissue imaging at a microscopic scale, with characteristics comparable to traditional histological slides, and with a label-free modality. This raises the possibility of an `optical biopsy’ to aid clinical decision making. This approach faces barriers for being incorporated into clinical practice, including the lack of existing images for training, unfamiliarity of clinicians with the novel image domains and the uncertainty of trusting `black-box’ machine learned image analysis, where the decision making remains inscrutable. In this paper, we propose a new method to transform images from novel photonics techniques (e.g. autofluorescence microscopy) into already established domains such as Hematoxilyn-Eosin (H-E) microscopy through virtual reconstruction and staining. We introduce three main innovations: 1) we propose a transformation method based on a Siamese structure that simultaneously learns the direct and inverse transformation ensuring domain back-transformation quality of the transformed data. 2) We also introduced an embedding loss term that ensures similarity not only at pixel level, but also at the image embedding description level. This drastically reduces the perception distortion trade-off problem existing in common domain transfer based on generative adversarial networks. These virtually stained images can serve as reference standard images for comparison with the already known H-E images. 3) We also incorporate an uncertainty margin concept that allows the network to measure its own coincidence, and demonstrate that these reconstructed and virtually stained images can be used on previously-studied classi cation models of H-E images that have been computationally degraded and de-stained. The three proposed methods can be seamlessly incorporated on any existing architectures. We obtained balanced accuracies of 0.95 and negative predictive values of 1.00 over the reconstructed and virtually stained image-set on the detection of color-rectal tumoral tissue. This is of great importance as we reduce the need for extensive labeled datasets for training, which are normally not available on the early studies of a new imaging technology.

Journal/Review: IEEE Access

Volume: 9      Pages from: 32081  to: 32093

More Information: This work was supported in part by the European Unionīs Horizon 2020 Research and Innovation Programme under Grant 732111 (PICCOLO project), and in part by the Basque Governmentīs Industry Department through the ELKARTEK Programīs Project 3KIA under Grant KK-2020/00049. The work of Andoni Elola was supported by his pre-doctoral research from the Basque Government under Grant PRE_2019_2_0100 and Grant IT1229-19.
KeyWords: Histopathology analysis, convolutional neural network, domain adaptation, optical biopsy,
virtual staining, Siamese semantic regression networks
DOI: 10.1109/ACCESS.2021.3060926

Citations: 12
data from “WEB OF SCIENCE” (of Thomson Reuters) are update at: 2024-10-27
References taken from IsiWeb of Knowledge: (subscribers only)
Connecting to view paper tab on IsiWeb: Click here
Connecting to view citations from IsiWeb: Click here