TY - JOUR
T1 - Improving Depth Estimation by Embedding Semantic Segmentation
T2 - A Hybrid CNN Model
AU - Valdez-Rodríguez, José E.
AU - Calvo, Hiram
AU - Felipe-Riverón, Edgardo
AU - Moreno-Armendáriz, Marco A.
N1 - Publisher Copyright:
© 2022 by the authors. Licensee MDPI, Basel, Switzerland.
PY - 2022/2/1
Y1 - 2022/2/1
N2 - Single image depth estimation works fail to separate foreground elements because they can easily be confounded with the background. To alleviate this problem, we propose the use of a semantic segmentation procedure that adds information to a depth estimator, in this case, a 3D Convolutional Neural Network (CNN)—segmentation is coded as one-hot planes representing categories of objects. We explore 2D and 3D models. Particularly, we propose a hybrid 2D–3D CNN architecture capable of obtaining semantic segmentation and depth estimation at the same time. We tested our procedure on the SYNTHIA-AL dataset and obtained σ3 = 0.95, which is an improvement of 0.14 points (compared with the state of the art of σ3 = 0.81) by using manual segmentation, and σ3 = 0.89 using automatic semantic segmentation, proving that depth estimation is improved when the shape and position of objects in a scene are known.
AB - Single image depth estimation works fail to separate foreground elements because they can easily be confounded with the background. To alleviate this problem, we propose the use of a semantic segmentation procedure that adds information to a depth estimator, in this case, a 3D Convolutional Neural Network (CNN)—segmentation is coded as one-hot planes representing categories of objects. We explore 2D and 3D models. Particularly, we propose a hybrid 2D–3D CNN architecture capable of obtaining semantic segmentation and depth estimation at the same time. We tested our procedure on the SYNTHIA-AL dataset and obtained σ3 = 0.95, which is an improvement of 0.14 points (compared with the state of the art of σ3 = 0.81) by using manual segmentation, and σ3 = 0.89 using automatic semantic segmentation, proving that depth estimation is improved when the shape and position of objects in a scene are known.
KW - 3D CNN
KW - Depth estimation
KW - Hybrid convolutional neural networks
KW - Semantic segmentation
UR - http://www.scopus.com/inward/record.url?scp=85125012408&partnerID=8YFLogxK
U2 - 10.3390/s22041669
DO - 10.3390/s22041669
M3 - Artículo
C2 - 35214571
AN - SCOPUS:85125012408
SN - 1424-8220
VL - 22
JO - Sensors
JF - Sensors
IS - 4
M1 - 1669
ER -