Avular is working on a product that uses radar and stereovision technology for the autonomous navigation of mobile robots and drones. Before our clients can start using this navigation solution, we need to prove that its software is safe. This is challenging, since no functional safety norm exists for software. Avular will adopt the RobMoSys approach to software development. This ensures that the programming work is carried out in a well-defined and neat way. Thereby limiting the likelihood of errors and allowing to estimate its performance, based on the formal model of the system.
Avular’s new navigation solution relies on the combination of radar data with stereovision data. As these technologies are complementary, they are the perfect match to generate reliable data in a large variety of circumstances, allowing for autonomous navigation and SLAM (simultaneous navigation and mapping). Thanks to the low hardware costs of this technology, this navigation solution will have a way lower price than existing LiDar-based solutions. On top of that, our new product can be used in bad weather conditions and dusty areas, which is not possible with LiDar technology. Based on viability study into this product, Avular sees a large application potential for this product, that has the potential to disrupt the mobile robotics market.
High quality and reliable stereovision cameras and radar chips can be bought off the shelf and serve as the starting point for this navigation solution. The way in which our engineers combine the output of these technologies is where the real magic happens. This is done through a complex sensor fusion approach that merges the point cloud data from the radar, with the more detailed data from the stereovision camera to obtain reliable depth maps of the robot’s surroundings. These fused depth maps are used for SLAM and autonomous navigation.
To ensure that robots that use this navigation solution navigate safely, we need to show that the software and the depth maps resulting from it are safe. normally, functional safety can be tested using safety standards, but these do not exist for software.
Safety and RobMoSys
In May 2020, Avular received funding from the RobMoSys project to find out whether the adoption of the RobMoSys approach would help in proving safety of a robotic system. RobMoSys provides an European ecosystem for open and sustainable industry-grade software development for robotics. It proposes a model-based approach for robotics, that allows to build better defined systems. This could help us to show that our navigation system is safe.
In the first stage of our project, we selected an existing machine learning network for sensor fusion on embedded platforms and adapted it for use in our use case. It was initially made to take two images from the stereovision camera to build a depth map and we made sure it could also handle radar data as additional input. In different stages of the network, the data would be fused, generating depth maps at different resolutions. For a machine learning network to work well, it needs to be trained. We did this with data from a publicly available dataset and collected data for a custom dataset as well. After training of the system, we could see that our improvements made the machine learning network faster and better in estimating depth than the original network.
Demonstration on cleaning robot
The next step was to convert our software into a RobMoSys conformant formal software model and to make it work on a real robot. We integrated our solution and a stereovision camera on our cleaning robot to see how it performed during operation. Due to low quality of our radar sensor, we only used the stereovision camera. Test runs showed that our solution can make depth maps on an embedded platform and in real time. Even though the quality of the depth maps is not yet perfect, these are promising results and Avular will develop this solution further.
A video of this demonstration is one of the public deliverables of this project and will be added to this page in December.
Functional safety has been an important topic in every stage of this project. We studied the safety standards and matched the recommendations and requirements from these standards with our development activities. More importantly, we kept asking ourselves the question how the adoption of the RobMoSys approach would help us in conforming to the functional safety standards and how RobMoSys could be expanded and improved to make an even better fit. These results are summarized in a safety report, which is a publicly available deliverable of this project and which will be posted on this page in December.
Subsidy: RobMoSys (Horizon2020, grant agreement No 732410)