Intelligent 3D Vision

Autonomous driving, 3D imaging and reconstruction, for example in medical technology, augmented realities, etc. are topics that are currently developing rapidly. As a result, the need for high-performance, specialized solutions for the three-dimensional capture of environments and objects is also growing. Often integrators and developers have to resort to less flexible COTS products which only cover different parts of computer vision or can only satisfy the needs of different environments and operating conditions to a limited extent.

Stettbacher Signal Processing AG (SSP) offers customized solutions for stereo and multi-camera applications. From the selection of optimal image sensors, hardware acceleration and AI integration to point cloud measurements and IP69K protection class, no wishes remain unfulfilled. The company also provides support for smooth integration into existing infrastructures, for example with a connection for ROS2 (Robot Operating System 2).

Stereo camera from Stettbacher Signal Processing AG in IP69K housing.

Application areas
Stereo cameras are used to create 3D models of objects and environments, for example to precisely control pick-and-place processes, or to help self-driving vehicles and robots better understand and navigate their environment. Together with other sensor data, such as IMU, radar, ultrasound, etc., autonomous systems are thus enabled to create high-resolution maps of new environments and to find their way around them. This is especially essential for subsequent localization and navigation. Precise spatial perception helps to avoid accidents and improve accuracy and effectiveness.

Intelligent camera: searching for, classifying and localizing objects (here feathers) in the image. (Image: Dan Cristian Päduret, Unsplash).

In augmented reality (AR), stereo cameras are also used to capture depth information of a scene, such as the location and position of specific objects. Conversely, AR applications use spatial information to correctly position virtual objects and to enable interaction with them. If the depth information can be obtained in sufficiently high resolution and quickly, it is possible to create highly immersive, interactive AR experiences. For example, virtual furniture can be placed in a room and aligned with real walls and floors when virtually planning a new home.
Stereo vision also has many applications in medical technology, particularly in imaging and 3D reconstruction. In dental medicine, for example, 3D models of teeth and jaws can be created. This is very helpful for planning dental implants, braces and other treatments. In surgery, for example, stereo cameras can be used to generate models of the face and body to better plan plastic surgery. In all these applications, stereo cameras enable the creation of precise and detailed 3D models, which have become essential for modern planning and execution of medical interventions.

3D model of a surface. (Image: D toi, Unsplash)

Technology
Stereo cameras use two synchronized and calibrated cameras so that a pair of images is always captured from two slightly different perspectives. By correlating the two images, the so-called depth image is calculated, which gives the spatial coordinates to each pixel. Thus, a 3D description of the visible space is created. In this space, objects can be searched for, classified, localized and even their orientation determined by means of AI processes.
The step to a 3D image is very computationally intensive. The stereo camera from Stettbacher Signal Processing AG uses specialized computers for this, for example from Nvidia. The data of the camera are impressive: Both image sensors deliver 5 MPixel at 35 frames per second, respectively 60 frames at Full HD resolution, always with global shutter. This means that there is no image distortion even in fast-moving scenes. Thanks to the waterproof housing (IP69K), use in harsh environments, for example in wet or dusty conditions, is made possible. It can even be used under water and in temperatures ranging from -40 to +60 degrees Celsius. The integrated Jetson Xavier NX computer from Nvidia performs more than 20 TOPS with 384 GPU cores. The power consumption is about 20 W. The computer is freely programmable and connected to the outside via an Ethernet interface with PoE++. All in all a product that is worth seeing!

Services
On request, Stettbacher Signal Processing AG supports its customers in the use of the camera or in the development of computer vision application, including application-specific hardware and/or software. In this way, every vision solution comes within reach.