Segmentation multi-capteurs de déchets métalliques pour tri robotisé

Abstract

During the last decades, the intensive exploitation of natural resources has known a significant growth due to the increasing need for raw materials. As a consequence, the environment has been subjected to a lot of damage. Modestly, by improving recycling and waste recovery, this master's thesis contributes to a partial solution that could slow down the exploitation of natural resources. Indeed, this thesis is carried out as part of the Multipick project which is aiming at creating an industrial demonstrator capable of sorting over 20,000 tonnes of metals per year at a rate of 16 scraps per second thanks to the use of robots and artificial intelligence. More precisely, the objective of this master's thesis is to study and assess the potential of deep neural networks to increase the performance of the actual waste characterization system operating inside a prototype version of the demonstrator called Pick-it. The innovation of this work lies in the type of input processed by the deep neural network: instead of using classical RGB images, the deep neural network is fed with multi-feature tensors composed of eleven 2D images staked together and containing diverse characteristics extracted from a dual X-Ray transmission sensor, a 3D ranging camera, and a hyperspectral camera. Leveraging these multi-feature tensors, a fully functional and optimized Mask R-CNN deep neural network is successfully integrated inside the pioneer Pick-it prototype. This model has been shown to achieve excellent results in terms of localization but reaches too low classification accuracy with the prototype to outperform the current waste characterization method. Nevertheless, the achieved results show great potential and unveil various directions of future work that could lead to the creation of a deep learning model outperforming the current characterization method and reaching state-of-the-art performance in the task of multi-sensor instance segmentation.