Hyperspectral Target Identification Using Physics-Guided Neural Networks with Explainability and Feature Attribution

Overhead long-wave infrared (LWIR) hyperspectral imaging (HSI), covering a wavelength range of about 8 to 14 μ m, is particularly well-suited for chemical and material identification because LWIR signals are strongly dependent on unique thermal emission arising from a material's composition. As such, LWIR HSI has been utilized for material identification in numerous applications [1] , [2]. As an alternative to traditional adaptive matched filter detection algorithms, deep learning (DL) models (e.g., neural networks - NNs) have shown promising results for target identification within LWIR HSI scenes [3] , [4] , [5] , [6]. However, models that ingest radiance values can suffer in performance or reliability due to cumulative errors from atmospheric compensation, incorrect instrument calibration, and scene background correction. An alternative approach is to recover a material's emissivity and temperature information and utilize the emissivity directly for material classification. Unfortunately, emissivity retrieval involves solving an ill-posed inverse problem that is sensitive to estimated temperature and atmospheric components. In this work, we explore the use of physics-guided neural networks to automatically retrieve emissivity as an intermediate output during material classification ( Fig. 1 ; full description in Section 3 ). We compare physics-guided NNs to black-box NN models for material classification. Here, we are interested not only in comparing the predictive accuracy of the two types of models, but also in evaluating the interpretability of each model and the value of auxiliary information. For instance, in the physics-guided NN, the retrieved emissivity enables users to more easily understand intermediate neural network representations, which could assist in many tasks such as identifying outliers or adding to confidence in the classifier predictions. To evaluate which features each model uses for its classification predictions, we use feature attribution methods (e.g., SmoothGrad using integrated gradients [7] ).