The Use of the Hololens in the Food Processing Industry

At Schouw Informatisering, we take great pride in our research and adoption of innovative technologies that are of valuable use to the food industry – now and in the future. Since the launch of the Dynamics 365 platform, Microsoft enables us as a partner to continuously work on innovations through the Power Platform. For example, our Blockchain project with the WUR and our Avocado Case. Another project we have focused on for the past 5 months Is the HoloLens and how it could be utilized in the food processing industry. Read more about the research project and the results below, conducted by Dennis van Leeuwen.

The practical applicability of the Microsoft HoloLens

‘For Schouw, I investigated the practical applicability of the Microsoft HoloLens for the food processing industry. Conducting this research was my last step towards graduating in Information Management, the master program I followed at Tilburg University. The purpose of the study was to determine the HoloLens potential for the food processing industry, since this industry is lagging behind in harnessing the power of information technology. With this study I want to nudge food companies a step closer in embracing technological advancements such as the HoloLens.

The HoloLens: A Mixed Reality Device

The HoloLens is a Mixed Reality device that is already disrupting multiple industries. It has the ability to superimpose (3D) digital objects on top of the user’s view of the real world. Since the HoloLens is scanning and mapping the user’s direct environment, the device is able to locate the digital objects in the real world at the correct location, scale, and orientation. The user can interact with these digital objects through hand gestures or voice commands. The HoloLens can operate untethered, but connecting the device with services in the cloud or back-end information systems will make the device more powerful.

The applicability of the HoloLens for the food industry

The food industry is characterized by a high variety of products and processes, making it hard to generate a statement about whether the HoloLens is practically applicable for the entire industry. Because of the high variety of products and processes, I developed a Mixed Reality Assessment model to help form a first impression of the potential of the HoloLens to support a food industry work task.

Within this model, a work task is assessed based on multiple criteria. Depending on the outcomes of this assessment, possible solutions, utilization limitations and eventually the potential of the HoloLens to support the work task can be determined.

Testing the model in different food industry settings

Next step? Testing out the model in different settings. The first one being a hygiene inspection task. During the inspection, the HoloLens can guide the task performer through the task by showing the points of interest of the hygiene inspections in their field of view. The HoloLens can then indicate where to look and point out the critical points of the inspection. While looking at an inspection point, the HoloLens is able to automatically make videos or photos and store these directly in a back-end database. This will increase the traceability of the hygiene inspection – which is an important factor in the food industry.

In a second example, a vegetable inspection was assessed. Here, the view of the HoloLens can be coupled to an artificial intelligence algorithm in the cloud, which can be used to determine the quality and deviations of the vegetable based on image recognition. A technique we already developed in our Avocado Case. The algorithm can be trained based on the choices the vegetable inspectors make. The HoloLens can help by making the right choices by checking for deviations on the vegetable by comparing its live view with pictures in a database and giving suggestions about the classification of the vegetables. The algorithm will learn from false suggestions and correct them next time a classification suggestion is made. This will help (especially inexperienced) inspectors to shorten their learning curve and make the right decisions. Even a connection with the production planning is possible. Vegetables with a considerably short shelf life can be moved in the planning of the production process automatically, reducing food waste.

Next up: a business case?

Based on the results of the study, the MR assessment model is a useful tool to get a first impression of whether a HoloLens has potential to support a work task. For now, the next step is to decide if Schouw wants to continue to work towards a business case.. Which, in my opinion, makes sense, looking at the great potential this study showed.’

Written by Dennis van Leeuwen