Abstract
MICAAI 2020 Intra-operative automatic semantic segmentation of knee joint structures can
assist surgeons during knee arthroscopy in terms of situational awareness.
However, due to poor imaging conditions (e.g., low texture, overexposure,
etc.), automatic semantic segmentation is a challenging scenario, which
justifies the scarce literature on this topic. In this paper, we propose a
novel self-supervised monocular depth estimation to regularise the training of
the semantic segmentation in knee arthroscopy. To further regularise the depth
estimation, we propose the use of clean training images captured by the stereo
arthroscope of routine objects (presenting none of the poor imaging conditions
and with rich texture information) to pre-train the model. We fine-tune such
model to produce both the semantic segmentation and self-supervised monocular
depth using stereo arthroscopic images taken from inside the knee. Using a data
set containing 3868 arthroscopic images captured during cadaveric knee
arthroscopy with semantic segmentation annotations, 2000 stereo image pairs of
cadaveric knee arthroscopy, and 2150 stereo image pairs of routine objects, we
show that our semantic segmentation regularised by self-supervised depth
estimation produces a more accurate segmentation than a state-of-the-art
semantic segmentation approach modeled exclusively with semantic segmentation
annotation.