rFpro develops new technique to simulate the perception of vehicle sensors

UK simulation software specialist, rFpro, has developed a new technique
that helps to overcome the safety concerns around testing and developing
connected autonomous vehicles (CAVs) on public roads. By developing a physical model of the real world, known as the Ground Truth, rFpro can accurately test the vehicle's perception of its surroundings, which has not been previously possible.

This will enable legislators to define an approval process for a vehicle within a completely virtual environment, certifying it as safe to use on real roads. The technique is already being used to validate vehicle safety in Euro NCAP tests.

“Most system modelling begins with ideal sensor models in order to validate the algorithms and control systems of the vehicle, but this bypasses any limitations in the sensors themselves,” explained Chris Hoyle, rFpro's Technical Director. “Difficult lighting conditions, or the reflections in a shop window can corrupt a sensor's perception of the vehicle's surroundings, leading to potentially catastrophic errors. Thorough validation of a CAV or ADAS-equipped vehicle must include the sensors' ability to recognise and characterise the features of its environment.”

The ability to evaluate the sensors' perception during simulation matters because future legislation is likely to dictate the virtual testing and approval of any autonomous system before its use on public roads is permitted. The whole system must therefore be tested in a fully-representative virtual environment, not just elements of it.

Due to the vast number of miles required to validate an autonomous vehicle in a huge number of different environments, it is not feasible to do this in the real world. Sensor perception is the most challenging aspect because it requires a physically accurate virtual world with high levels of correlation to the real world and physically modelled sensors.

“Physical modelling means simulating the materials and properties of every object encountered by the vehicle and its onboard sensors, rather than just an abstract representation of it as used by most systems,” said Hoyle. “With several years' experience of creating digital twins of city streets, rural roads, proving grounds and test tracks, we understand the complexities of modelling features like changing weather conditions or road surfaces.”

“Our engineers are constantly being challenged by our customers to bridge the gap between simulated and real-world testing. Whether this is 8 stereo, 4K cameras with live exposure control and real motion-blur modelled, or even a radar model picking up the micro-doppler from a pedestrian moving their arm. All of this must be possible for successful simulation and can all be done using rFpro,” he added.

Hoyle believes rFpro is unique in providing closed loop end-to-end simulation of autonomous systems, with a vital element of this process being the inclusion of an accurate vehicle model that responds in a fully representative manner to road surface changes and control inputs.

“Perception is critical not just from a safety point of view but also for consumer satisfaction,” he said. “For example, with our system, minute road surface differences are accurately modelled, which means ride comfort can be assessed. We can explore the CAV's ability to identify and avoid potholes, like a human would. Without this, passengers are unlikely to want to ride in an autonomous vehicle again.”

A current UK government project, dRISK, uses rFpro software to validate sensor models against the real world so they can be correlated with actual sensors. This is seen as an essential stepping stone towards end-to-end validation of autonomous systems through simulation.

“Due to the infinite inputs possible from these sensors, there is a big emerging need for autonomous system test engineers making use of simulated software,” concluded Hoyle. “Previously there has been around a 1:1 ratio of software engineers to test engineers in the automotive sector. We believe this is moving closer towards the avionics sector where there are around 5 test engineers per software engineer to ensure the safety of passengers.”