Filling Voids in Elevation Models Using a Shadow-Constrained Convolutional Neural Network

We explore the use of convolutional neural networks (CNNs) for filling voids in digital elevation models (DEM). We propose a baseline approach using a fully convolutional network to predict complete from incomplete DEMs, which is trained in a supervised fashion. We then extend this to a shadow-constrained CNN (SCCNN) by introducing additional loss functions that encourage the restored DEM to adhere to geometric constraints implied by cast shadows. At the training time, we use automatically extracted cast shadow maps and known sun directions to compute the shadow-based supervisory signal in addition to the direct DEM supervision. At the test time, our network directly predicts restored DEMs from an incomplete DEM. One key advantage of our SCCNN model is that it is characterized by both CNN data inference and geometric shadow cues. It thus avoids data restoration that may violate shadowing conditions. Both our baseline CNN and SCCNN outperform the inverse distance weighting (IDW)-based interpolation method, with the shadow supervision enabling SCCNN to obtain the best performance.