Industrial image processing is usually the domain of specialists, because everything the camera is supposed to “know” or do has to be coded. The advent of artificial intelligence brings something new here – not only in terms of technology, but also with regard to users. By using a system like IDS NXT, which covers all workflows from taking training images to running the neural network on a camera, it is possible to make camera technology accessible to people in other professions. “I am an engineer no software developer and I can say now that I have done image processing with AI”, explains Marco Ullrich from Kempten University of Applied Sciences. As research associate at the Institute for Production and Informatics, he supervised a group of students who worked on the development of a mill-playing robot with AI in the course of their master’s degree. This is an interview about one of their latest projects.

Could you explain the subject and purpose of this project?
The project’s goal was to develop robots that could play the German board game “Mühle” (Mill) geographically independent. This means that one robot cell in Kempten (Germany) and another in Salzburg (Austria) could play against each other. We achieved this by using a vision system to detect changes on the game board. The intelligent IDS NXT camera was an essential component of this system.

What challenges did your team face during the project work?
The initial situation was to replace an existing Raspberry Pi camera with an industrial camera. We also needed to select a suitable image processing algorithm, lighting, and camera installation position. Additionally, we had to find a method to equalise the image for an HMI (Human Machine Interface that allows users to communicate with machines), as we installed the camera at an angle but required a top-down view for the user interface. Overall, the integration of the camera into the existing robot system was a significant challenge.

What was the specific task of the vision system in your application?
The vision system’s task was to detect changes on the game board reliably and safely. As engineers and not software developers, we also intended a system that is user-friendly and easy to use. The camera needed to be suitable for industrial use and ideally innovative or state-of-the-art. In addition, we wanted the image processing to be done using artificial intelligence. However, since our target group was not necessarily composed of AI experts, we needed to make sure also the AI application was easy to handle and understandable.

Why did you choose IDS camera technology for the project?
The IDS NXT system was ideal for our project because it met all our requirements. The first and most important factor was the focus of IDS on reducing the barrier of using artificial intelligence in real applications, making it user-friendly and accessible to engineers like us who are not software developers. Furthermore, we did not require additional computing resources for image processing, making it easy to integrate into our existing system.

Could you explain why the IDS NXT system was easy to use?
As I previously mentioned, I am an engineer, not a software developer. However, I was able to work with AI and neural networks for image processing through the IDS NXT system. It does not require any large investments in computing resources for image processing since everything happens on the camera device itself, eliminating the need for any additional processing unit. This made it incredibly easy to integrate the system into our existing setup.

Anyone who wants to test the potential of cameras and AI for their own applications should take a closer look at the IDS NXT Experience Kit offered by IDS Imaging Development Systems. From camera to software licence, it contains all the components needed to create, train and run a neural network on IDS NXT cameras. The use of Deep Learning-based image processing can therefore be evaluated and realised in a short time.