Hyperspectral Imaging is a technology in which a camera collects an image of a target. The sensor records reflected light from the visible and near-infrared regions in several tens or hundreds of narrow spectral channels, also called wavelengths. How light interacts with an object is called the study of light, scientifically known as (imaging) spectroscopy.
Each material reacts differently with light. When we measure this interaction, it results in a unique individual ‘fingerprint’ called a spectral signature. A spectral signature holds unique quantifiable information in each wavelength that we can use to identify objects and extract information.
Tip for the busy ones: there is a handy navigation pane on the left side. Use it to move straight to the section that interests you. 👀
With Hyperspectral Imaging you can reveal an object's hidden information, identify invisible objects and features, and separate similar-looking objects.
Optical sensors, such as your phone camera, emulate the human eye by capturing reflected light in the red, green, and blue (RGB) spectral channels. Sensors form a photograph of RGB pixels that human eyes can process. RGB spectral regions represent the visible light spectrum. Humans use visible light to quickly detect objects' physical properties (size, color, and shape) and their relation (proximity and distance) to their environment.
If we want to know an object’s chemical composition, what it is made of, and other chemical properties, simple RGB channels are not enough. RGB sensors belong to the group of multispectral sensors, which do not have continuous spectral channels but only use a subset of selected wavelengths.
For most applications, multispectral channels are not sensitive enough to accurately identify specific object features. Simply put, the spectral resolution of RGB cameras is too low to detect spectral features or the spectral features are only visible to near-infrared sensors.
An illustrative spectral profile of two plastics, polycarbonate (PC) and polyvinyl chloride (PVC).
Hyperspectral imaging, however, can reveal an object's hidden information, identify invisible features, and separate similar-looking objects. For example, when thinking about a food production line, the human eye or a normal camera is not accurate enough to detect, if the food package is airtightly sealed. The narrow bands of HSI sensors can detect water content, potential diseases, or chlorophyll content in leaves which are invisible to RGB sensors.
HSI is not just for large academic research projects and corporations.
HSI has been applied in a wide range of applications and industries: to detect rock minerals from airborne images, plant diseases in agricultural crops, eye anomalies in biomedical imaging, foreign materials in food processing, and in surveillance using thermal HSI for detecting hidden enemies.
Sensor prices are rapidly decreasing. The performance-to-cost ratio meets the industry standards, allowing small to medium size companies to invest in automated quality inspection setups. In addition, sensors are becoming more versatile. New sensors are introduced for specific conditions; from lightweight sensors for drones, robust airplane compatible sensors, to sensors for industrial applications requiring real-time data collection.
Limitations of Hyperspectral Imaging
Being able to detect minuscule differences because of chemical differences makes Hyperspectral imaging incredibly powerful. However, there are also limitations.
The amount of data collected with HSI is much larger compared to conventional RGB setups. This increases latency and makes the data slower to process. However, this can be improved by testing, which spectral bands are the best to use in a specific case and then limiting the number of spectral bands from e.g. 128 to 8. That way, HSI is also suitable for fast production lines.
Hyperspectral imaging is useful for applications in which the object of interest can be distinguished by its surface properties. Thus, color matching and finding foreign materials across a product stream make good candidates. Unfortunately, HSI can not penetrate the surface of an object. Applications, where we need to look inside products (such as bone detection in meat processing) do not work.
Hyperspectral Imaging has many benefits over conventional RGB camera setups and is rapidly gaining interest from the private sector.
Using hyperspectral imaging has many advantages, but what are the main reasons behind its growing popularity?
Hyperspectral imaging is not only reserved for corporations and large academic research projects. Sensor prices are rapidly decreasing. The performance-to-cost ratio meets the industry standards, allowing small to medium size companies to invest in automated quality inspection setups. The need for manual inspections and laboratory testing decreases, which results in outstanding cost savings.
Hyperspectral imaging is a great asset to quality assurance since the system can be automated to inspect object surfaces to identify possible anomalies or deviations. It works on different surface properties from painted materials, to wood, plastic, and metal. Another use case of hyperspectral imaging for quality assurance is to make sure that all objects are correctly placed, positioned, and packaged during the process. When combined with AI, the system can autonomously react to deviations.
Many traditional methods use destructive methods to find the target’s chemical properties. HSI is an optical sensor and requires no physical contact with the target. The target does not need to be probed with cables or sensors nor physically be prepared or altered - no waste is generated.
Using hyperspectral imaging is completely harmless. The sensor is passive and it captures the target’s reflected light from a light source such as a lamp, or the sun. HSI can safely be used on humans and is gaining popularity in the medical field. For example, HSI can be used to diagnose potential eye diseases.
Spectral sensors are very versatile. The same sensor can be applied for multiple applications, from identifying plastics in food, to detecting moisture content in crops. Spectral sensors do not require the target to be in close range. Sensors can be mounted on moving vehicles such as drones, and airplanes to cover large areas quickly. HSI is used frequently in forestry and agriculture to monitor and assess forest and crop health for diseases.
Hyperspectral sensors are very reliable and capture imagery at very high speeds. Depending on the type of sensor these sensors can capture up to 6510 frames per second. This is more than enough for any industrial application that requires imaging of fast-moving objects. HSI sensors are ideal for industrial applications that require real-time inspection for in-line conveyor belts with fast-moving objects.
One of the major benefits of Hyperspectral is that the sensor operator does not need any subject matter experience to record the target’s reflectance. An entire spectrum can easily be acquired at each point. All spectral and spatial information can be mined during the postprocessing phase.
With hyperspectral imaging, it is possible to detect alien objects in food processing, inspect the food quality on a highly detailed level and identify holes in packaging.
The most common detection methods used for food inspection are metal detectors and x-rays. Metal detectors and x-rays are very good at detecting metal objects, but they lack the ability to detect biological properties such as fat-, water content, or meat tenderness.
To ensure food safety, any foreign material that enters the food processing production line needs to be detected. Traditional food quality inspection includes tedious food sample laboratory testing with results coming back the next day or later.
Other HSI use cases are detection of discoloration, identification of insects, shell pieces, stones, wooden objects between candy, predicting seed and grain protein content, measuring ripeness in fruits, and reducing product waste.
Food manufacturers have implemented hyperspectral sensors to ensure and improve product consistency, and to increase yields by reducing food waste in a systematic, reliable, and data-driven way. In this section, we will present three well-tested applications of Hyperspectral Imaging in the food manufacturing and processing industry.
Hyperspectral imaging can be used to inspect food quality with high precision. The reflectance spectra can be used to rapidly assess food quality, such as ripeness, or identify anomalies, such as food molds. HSI can answer questions such as: how much mold is present on food samples, and where the food mold is located.
HSI enables real-time, reliable, and precise detection of foreign objects. When HSI is combined with machine learning algorithms, HSI sensors enable digital sorters to automatically detect and remove foreign material. The rapid and precise detection of HSI sensors detecting foreign objects in bulk food streams saves time and money when compared to traditional laboratory testing. As surprising as it may sound, objects such as plastics, glass, wood, rubber, stones, metal, and other objects are common examples found in food production lines.
A typical use case for Hyperspectral Imaging in the food industry is detecting holes in food packaging. Food packages are heat-sealed but holes can occur during this process. Food packages need to be airtight to minimize the risk of food contamination. When air enters food packaging through holes, bacteria will rapidly deteriorate food quality and grow molds and fungi. Incorrect heated sealed food packages will deteriorate food quickly and cannot be sold to the end-user, resulting in revenue loss for the manufacturer.
Hyperspectral Imaging can reach 99% classification accuracy
With hyperspectral imaging, raw material inspections can be automated. Industrial quality inspection of fast-moving objects requires sensors that can operate and make assessments in real-time. A computer vision setup combined with Hyperspectral Imaging (HSI) sensors and machine learning algorithms allows for automatic assessment at high speed and accuracy.
Industrial HSI classification applications often reach classification accuracies above 90%. In combination with object detection methods and pixel clustering, accuracies can reach up to 99%. Industrial HSI setups calculate statistics of the product flow classification. Classification statistics, product quantities, product pass or fail, and other important metrics are logged and can be sent to robots up the stream to automatically remove bad products from the production flow.
Product classification and verification are where Hyperspectral Imaging really shines. The dense spectral information of HSI sensors makes it possible to detect product features, caused by water molecules, and verify correct assembly. Product verification applications can range from detecting mold in food products, detecting counterfeit medicines, detecting plastics, to detecting tree species variation in woodchips.
For example, the car-, chemical-, and textile industries require thorough testing of surface colors to exactly match the predefined color standards. Any deviation in surface color will result in returned products and unhappy customers. Flaws can result in entire product batches being discarded causing major losses. Identifying small color deviations, or product features caused by water molecules with the naked eye or RGB cameras is nearly impossible and can only be done with HSI cameras. Hyperspectral imaging can automate tedious human inspection and produce reliable and accurate product inspection, improve product grading, reduce waste, and increase yield.
The processing industry has started to shift to data-driven decision-making. The increasingly competitive landscape and the complexity of product manufacturing have pushed manufacturers to look for new methods to stay ahead of their competition. Manufacturers have started to incorporate new types of sensors and collect vast amounts of quality inspection data. Hyperspectral imaging has evolved from the secretive airborne military applications and white coat laboratory testing environments to the factory floors. With decreasing sensor costs HSI setups are now slowly making their way into production lines.
Read also this blog: How Hyperspectral Imaging and AI can boost your quality and reduce quality costs
Computer vision techniques and the use of different sensors are an integral part of the waste- and recycling processing industry.
Waste recycling plants depend on an optimized system of sensors identifying, separating, processing, and managing a wide range of materials. Waste is collected, processed, and recycled by machinery into reusable raw resources, and finally used in consumer products. To optimize the recycling processing chain, waste processing plants separate waste based on quality. Spectral cameras offer a solution to quickly detect plastic types, help in separating and sorting waste and detect hazardous materials in building waste material.
X-rays, gamma rays, and radio frequencies among others have been used to assess biomass quality but they are not safe considering human health. HSI provides a scalable, rapid, and accurate solution to reduce waste and optimize waste management processes.
Plant managers often lack insights into the biomass composition of organic waste (woodchip vs branches vs peat), moisture content, and woodchip tree species type. Woodchips are disposed of on conveyor belts in waste processing plants and then transported to burners to produce gas. Woodchips with too high moisture content increase gas emissions which decrease the burner energy yield. Machinery in downtime can result in hundreds of thousands of euros in annual revenue loss, and the knowledge of moisture content can help to avoid it.
We can use HSI sensors and machine learning algorithms to identify and separate waste materials. The near-infrared (NIR) region is ideal for separating waste since organic waste reflects the most light in the NIR spectral region. (The NIR region refers to the 700 and 2500 nanometer spectral region and is very suitable to detect woodchip tree species and moisture content.) NIR radiation is safe for humans and can measure multiple vegetation properties at once. In the recycling industry, Hyperspectral Imaging sensors are used for identifying and sorting types of industrial plastic, polymers, black plastics, and rubber material at high framerates that meet industrial standards. Hyperspectral sensors provide an alternative to the traditional waste detection methods in waste processing plants.
If you read this far, you probably already wonder how Hyperspectral imaging could work as a part of your analytics and quality assurance setup.
During the first meeting, we discuss the challenges you have and if Hyperspectral Imaging is a suitable technology for your use case. Once we have found out what the main challenges are, and we see the business potential for an HSI application, we proceed to the solution design phase.
After we have identified the business requirements and see a possible HSI use case, we move on to the solution design phase. The design phase is about identifying what resources are needed, creating an initial cost and time estimate, listing all assumptions, identifying the possible outcomes and limitations of the application, and deciding about the best approach to take. We break down each requirement into small easy-to-manage parts and create a well-designed custom plan of action.
Every Advian project follows a tailor-made procedure. The three-step procedure includes 1) Proof of concept, 2) Production environment testing, 3) Full-scale deployment in the production environment. Based on your needs, Advian will come up with a plan that includes our strategy, a detailed cost analysis, deliverables, and a schedule.
During the proof of concept (PoC) phase, we do a quick feasibility study in a laboratory-style setting. The PoC phase is a relatively inexpensive way to quickly understand the major challenges and overall project feasibility. We generate quick results and have performance metrics that will give you a go or no go on the project.
When the results of the PoC show project feasibility, we proceed to the next phase. This phase involves the testing of collecting spectra of high-volume samples at high speed in a production-like environment. This phase allows us to optimize algorithms and understand the biggest challenges of the HSI setup, working at full speed and with a variety of samples. The results of the high-volume sample statistical testing will provide you with accurate performance metrics. We will provide you with a detailed report including a performance report and cost analysis, which allows you to make a well-informed decision on the next steps.
The third and final phase is the full-scale deployment phase. During this phase, we will make sure all the hardware and software are robust and thoroughly tested. The final installation on factory machines or conveyor belts will be done with your own technicians or by a certified third party. If required, we will build a user-friendly custom monitoring dashboard based on your needs with summary statistics and other important metrics to enable continuous monitoring.
Running the solution in a production environment will create insights impossible to get anywhere else, which will help you to improve the solution. At this stage, we can validate the original business case and create a scale-up and development roadmap for the future.
The last thing to do is to capture the full business value by scaling up the solution in your operations. We will provide needed training or maintenance services, so you can focus on running your business.