DVN: Michael, you have worked at Ibeo for 24 years, meaning you have been there from the early stage just after Ibeo was founded in 1998. If you look back to those days and compare with today: what has changed essentially? Which key steps did the company and the lidar technology experience in those 24 years?
Michael Kiehn: 24 years is a long time in which a lot has changed. This extends to technology, the market and competition.
Ibeo Automotive Systems was founded as a subsidiary of Ibeo Holding under the name Ibeo Automobile Sensor in June 1998. At that time, Ibeo was already developing, building, and selling lidar sensors, with applications in various areas in the industry. Then, hardly anyone thought about lidar at vehicles on the road, but nevertheless Ibeo Automobile Sensor set out to bring the existing lidar technology to the car.
To do this, we first took Ibeo’s existing mechanical scanning devices and developed object detection and tracking algorithms for them. These lidar sensors were designed for indoor use and only on slow-moving vehicles and they were a far cry from the 3D lidars we know today. Accordingly, the sensors also required work to make them suitable for use in cars. For example, they could only scan in one plane. If the car did pitch movements or load conditions changed it could happen that objects were lost, or the ground was detected as an object.
An important technical step was to develop lidar sensors that could measure in multiple planes. These were still not 3D lidars as we know them today, called them 2.5D then. The planes were sufficient to compensate for the pitching movements of the vehicle to a certain degree and to recognize ground detections as such. Another disadvantage of the devices was that they could only detect a single reflection by laser shot at that time. When used indoors, this is perfectly adequate. Outside, however, this leads to some problems. For example, raindrops do lead to detections. If a raindrop is detected, the relevant object can no longer be detected. Under certain circumstances, even a slight soiling of the windshield can lead to detections, so that nothing but the windshield can be detected any longer.
Consequentially, another logical step in improving the technology was to develop sensors that could process more than just one echo. These two improvements basically made lidars usable in automobiles in the first place. Large part of these developments took place in projects funded by the European Commission. Products like the Ibeo LUX or Valeo’s Scala, which is derived from the LUX, are basically based on this technology we developed that time.
Ibeo had applications for lidar sensors in a wide variety of areas, such as surveying the instrument landing system at airports or the semi-automatic unloading of container ships. Sensors for personnel protection in manufacturing plants or warehouses were also an important area of application. However, there was no market for automotive lidars, at that time.
Ibeo was able to deliver a few sensors as prototypes for development to OEMs’ or Tier 1s’ research departments and sold them to universities and research institutes. In the late 1990s, early 2000s, we assumed that lidar could be used for Adaptive Cruise Control, which was a new application at that time. Apart from Ibeo, Hella together with Spiess was also working on lidars for this functionality. Their approach, however, was much more reduced than the scanning systems that Ibeo offered. It was based on 12 fixed measuring beams. With that Hella even achieved a nomination with Chrysler. Eventually, however, radar prevailed as the sensor for Adaptive Cruise Control, and lidar became an afterthought.
In 2009, Hella stopped lidar development and Ibeo was the only company left on the market for Automotive lidar. The turning point, however, also falls in this period. The DARPA Grand and Urban Challenges revived interest in automated driving and thus in lidar sensors. Since then, it feels like hardly a week goes by without a new lidar startup being founded.
A milestone for Ibeo was the nomination of Valeo as supplier for the first lidar sensor at Audi. We were able to support Valeo in developing the Scala as a sensor for automated driving. Even though the number of units for this first generation remained manageable, further nominations for Valeo also with the second and third generation show that a breakthrough is achieved here and lidar seems to be finding its place in the automotive market.
DVN: For the outside automotive world, a key cornerstone was the design of Scala 1 together with Valeo based on Ibeo technology, which was launched in 2018. Can you give us a rough summary, which hurdles Ibeo had to overcome and how much time it took to bring this product to market maturity?
MK: The biggest hurdle was making the sensor mass producible. This had a significant influence on the design of the device. The LUX was designed to produce 1000 units per year. Now, however, it had to be manufactured more than 300,000 times per year. No one had done that before, so in addition to the design, a fundamental development of the manufacturing processes and tools was necessary. It was fascinating to see how the quality of the sensors continuously improved.
DVN: Currently, you are presenting your second product generation called ibeoNEXT, a full solid-state design. What are the key specifications and engineering elements differentiating ibeoNEXT from Scala 1?
MK: The biggest difference in the specification is that the first-generation Scala could capture a few tens of thousands of points per second, while the ibeoNEXT is capable of capturing a few hundred thousand points per second. This is reflected in the fact that with the ibeoNEXT a true 3D point cloud can be generated, and significantly more details can be captured. In addition to distance and angle information the IbeoNEXT sensor provides intensity information on the measurement points as well. So, it is in fact a 4D sensor
DVN: Some more detailed technical questions. As far as we know you have made the decision for 885 nm emitter wavelength. Why not use 905 nm or even 1550 nm?
MK: There is a general debate in the business whether Near Infrared (NIR, ~900nm) or Short Wave Infrared (SWIR, ~ 1550nm) is the best choice. There are some advantages of SWIR like the much higher eye safety limits that would allow for larger range of the lidar. But it is our estimation that the available technology is not yet competitive with respect to maturity and cost. Therefore, we have decided for NIR.
In NIR, the most commonly used wavelength is 905 nm. When using silicon as material for the detectors –which is favourable because of cost- then one must accept that the sensitivity drops with longer wavelengths. When we started the development of the ibeoNEXT, the SPAD detectors we planned to use were not optimized for larger wavelengths. So, the choice was to stay with wavelengths as short as possible. This ‘as possible’ is actually limited by the ability of humans to see light. It is assumed that humans can see light up to ~700nm. This is true when it is about clearly seeing something. When it comes to just seeing something deep red, then there are people that can see much longer wavelengths. Therefore, we concluded that 885nm was the shortest possible wavelength to be used.
DVN: The power consumption of lidar systems is estimated to be 7-10 Watts per lidar sensor. This is comparable to the consumption of a simple LED headlight. Will power consumption become a key criterion for application of lidar sensors?
MK: I won’t say it is actually a key criterion. But thinking about electric vehicles, each consumer on board is an issue with respect to battery lifetime. This is basically true for any kind of equipment. On the other hand, there is the requirement to have a very small package size for lidar sensors. This is to enable better integration into the vehicle. But this also means that the power dissipation needs to be transported out of the package and the smaller the package the higher the temperature will rise inside. So, power consumption is at least a critical parameter for the design.
DVN: ibeoNEXT will provide 4D data, the fourth dimension being luminance of an object. What is the added functionality this feature will generate compared to 3D data only?
MK: This feature also turns the lidar into a camera. Today, the resolution of lidar sensors is low compared to cameras, but it is still amazing to see the degree of detail which can already be detected in the intensity images of these lidar sensors. What we can do with this now is to support object classification. When resolution of the lidar sensor increases in future generations this feature might be used also as kind of an infrared camera e.g. as a redundant path for reading traffic signs. In fact, this would be kind of an active camera, coming with the advantage that it is largely independent of the ambient light.
DVN: ibeoNEXT is a modular concept which can be adapted to different FoVs and ranges by changing the optics. What is your estimation about the number of near range/mid-range/long range lidar sensors needed by the end of this decade on a premium vehicle?
MK: Of course, this depends on the requirements of the specific application and usually the system architecture of the vehicle is up to the OEM. Today, we have a front-looking system consisting of three to four ibeoNEXT sensors. Two of them for the mid-range and one to two for the long-range. In the future, this will have to work with two sensors, one for long range with a limited field of view and one for mid-range with large field of view.
Furthermore, I think we will have one short-range sensor on each side of the vehicle to monitor the entire side of the vehicle. To the rear, I think the vehicle will get by with a single mid-range sensor. For a lane change function, we could think about a single long-range lidar in the rear as well. However, in my view, that makes little sense. Unlike the long-range sensor in the front, which is essentially intended to monitor its own lane, the aim in at the rear is to monitor also the adjacent lanes. With a single long-range sensor in the middle of the vehicle, however, it has to be considered that the adjacent lanes can be obscured by vehicles driving behind the ego vehicle.
If the high resolution of the long-range lidar is required for the lane change, e.g. to be able to determine on which of these lanes a vehicle is approaching from behind when there are several adjacent lanes, then two long-range sensors one at each corner of the vehicle must be used. A lidar sensor system could thus consist of five to seven lidar sensors towards the end of this decade.
DVN: It was announced in 2021 that ZF will produce the ibeoNEXT sensor in their factory located in Brest, France. When will/has been production start? Will it be for the global market or Europe only?
MK: Our current plan is to start series production during the first quarter of 2023. Initially, we will serve the global market from Brest. However, it is planned that at least the final assembly of the sensors can also take place at other manufacturing sites. This will be of particular importance for Asian customers.
DVN: What will come after ibeoNEXT? In an earlier communication with one of your colleagues, we got the answer ibeoNEXT NEXT. Maybe you can give us some more information about future plans?
MK: I think my colleague has found a good answer. But let’s take a look at what will make up the next generation of the ibeoNEXT. We will certainly continue or further develop the solid-state concept started with the ibeoNEXT. This means that the next generation will also be completely without moving parts. We see this as a clear advantage for the robustness of the product but also an advantage for its manufacturability.
What will change is the performance of the sensor. The aim is to cover a larger field of view with a single module and at the same time achieve long ranges with high angular resolution. In order to offer customers a good solution for level 3 – 5 applications, we expect to increase the number of pixels from today’s ibeoNEXT by up to a hundred times. In terms of range, the upcoming generation will surpass the ibeoNEXT by a factor of 1.5 to 2.
DVN: If you look to the global lidar sensor landscape, significant investments and venture capital is flowing into companies and start-ups in US and China. What will be the strategy of European companies to keep their position?
As I mentioned above, hardly a week goes by without a new start-up popping up somewhere in the world announcing the next great leap forward in lidar technology. These ideas generally sound very interesting, but they lack concrete proof of their transferability into the real world and in particular into automotive application and mass manufacturing. Therefore, I believe a viable strategy for European players in the lidar field is to drive proof of the functionality and robustness of our lidar technologies, through defining common standards for testing and rating lidar sensors with respect to their performance but also beyond performance with respect to robustness durability and their readiness for the intended application. Furthermore, it is paramount to have the critical production infrastructure in place with which you’re able to ramp up production volume. Ibeo is currently taking the necessary steps to finalize our production line at ZF Autocruise in Brest, France, which will be a huge milestone on our way to the start of series production in Q1 2023.
Additionally, I believe that networking opportunities such as the DVN Think Tank are important tools to exchange ideas with other players in the industry, gain insights into new approaches and technological advances presented by research professionals, and thus strengthen the market position of all participants.