(D-133) Hyperspectral Imaging Applications of Deep Learning

Introduction:: The goal is to grow and identify marine microbes that can form a smooth, stable biofilm to reduce drag on unmanned underwater vehicles (UUVs) to levels comparable to a hydraulically smooth state. Hyperspectral imaging (HSI) allows for us to image the target sample and collect both spatial information (X,Y) and spectral information (λ) and generate a 3D dataset which is referred to as a hypercube.

The Whiskbroom HSI method was used to determine the presence of the bacteria as well as what type(s) of bacteria are present. The Whiskbroom method is considered the slowest of all the methods due to the slow image acquisition time. This is due to the image being acquired point by point in millimeter intervals along with each point being imaged at varying wavelengths. While this does slow the image acquisition down, it allowed for us to obtain the most data per point for the deep learning algorithm.

The hypercube that is generated from the recorded reflectance can be processed through an image processing software. For this project, Fiji was used because there is a package in Python which allows for Fiji to be used with Python libraries such as NumPy, Matplotlib, and Seaborne. Once the images had been analyzed with Fiji, the data generated was put into an unsupervised deep learning algorithm because we wanted to be able to put in unlabeled inputs (the various bacteria) to train the model to help with the bacteria predictions.


Materials and Methods:: The setup used involved a super continuum source from 800 to 900 nm, a beam splitter, a diffraction gradient, various lenses, and a CMOS camera to record the data. The sample surface was scanned rapidly with hundreds of small bands of light which were then captured and recorded by the camera. Each sample scan used a different spectrum of wavelengths (such as between 800 and 900 nm) with narrow band passes. Once the image was recorded by the camera and saved, the images were then processed through Fiji which allowed for us to gather the intensity of the pixels at varying wavelengths. The peaks at specific wavelengths indicated the optical reflectance of the bacteria which help with identification.

Following the image analysis, the images were then run through an unsupervised deep learning algorithm. This algorithm was created using Python and used the PyImageJ package to import Fiji so it could be combined with the necessary libraries that included TensorFlow 2, Keras, NumPy, Pandas, Seaborne, and Matplotlib. Along with various statistical methods and functions, the algorithm was able to determine the type of bacteria present in the flow test setup and tell us what type was present.


Results, Conclusions, and Discussions:: With the imaging setup, various types of bacteria were able to be classified and identified by using Fiji along with deep learning. The images obtained are not collected in a way that a human could look at the image and understand what is being presented in the files because of the images only appearing as a narrow white line with varying pixel intensity. The images show both spatial information and spectral information (wavelength in nm) which can then be imported into an excel file as numerical values that can be used to train an unsupervised deep learning algorithm or be classified by the deep learning algorithm.

The goal of this project was to grow and identify marine microbes that were able to create a smooth, stable biofilm on UUVs to levels comparable to a hydraulically smooth state. The Whiskbroom HSI method was used along with Fiji and a deep learning algorithm to collect the data, process the signal obtained, and classify the signal as a bacteria based on the observed and recorded reflectance. While this method was only used to identify and classify some bacteria with a deep learning algorithm, the HSI method and deep learning can be expanded upon in the future with the detection, identification, and classification of diseased tissue, tumors, and similar ailments.


Acknowledgements (Optional): :

References (Optional): :