Rapid tracking of extrinsic projector parameters in fringe projection using machine learning

In this work, we propose to enable the angular re-orientation of a projector within a fringe projection system in real-time without the need for re-calibrating the system. The estimation of the extrinsic orientation parameters of the projector is performed using a convolutional neural network and images acquired from the camera in the setup. The convolutional neural network was trained to classify the azimuth and elevation angles of the projector approximated by a point source through shadow images of the measured object. The images used to train the neural network were generated through the use of CAD rendering, by simulating the illumination of the object model from different directions and then rendering an image of its shadow. The accuracy to which the azimuth and elevation angles are estimated is within 1 classification bin, where 1 bin is designated as a +/- 10 degrees patch of the illumination dome. To evaluate use of the proposed system in fringe projection, a pyramidal additively manufactured object was measured. The point clouds generated using the proposed method were compared to those obtained by an established fringe projection calibration method. The maximum dimensional error in the point cloud generated when using the convolutional network as compared to the established calibration method for the object measured was found to be 1.05 mm on average.