The EVIDENT project aims to address challenges in the automotive industry's validation and verification (V&V) processes for advanced driver assistance systems (ADAS) and autonomous driving (AD) features. Traditional V&V methods struggle to keep up with the increasing frequency of software updates. The project explores virtual validation strategies to complement or replace physical testing, thereby enhancing efficiency and safety assurance.

Automotive innovations are increasingly software-driven, necessitating frequent updates. Current validation processes heavily rely on physical testing, which is time-consuming and costly. The project focuses on how vehicle functionalities could be tested and validated in simulation models and what fidelity level that could be reached. By utilizing virtual environments, the project aims to proactively test software functions before deployment, ensuring accurate assessments of system performance in diverse scenarios.

The primary goal is to develop strategies that balance the realism of virtual test environments with practical implementation. Key research questions include:

• What level of realism is required for simulations to be credible for testing edge cases?
• How can virtual testing be integrated with real-world data to discover new edge cases?
• How can virtual testing ensure functional safety to satisfy regulatory bodies?

The project also seeks to establish metrics for comparing physical and virtual test results and to utilize open-source tools for broader industry use.

The project follows a structured approach:

1. Gap Analysis: Semi-structured interviews with industry experts were conducted to identify current best practices and challenges.
2. Simulation Toolchain Assessment: Each partner's simulation tools, and maturity levels were evaluated.
3. Scenario Development: Road network representations and test scenarios were developed using ASAM OpenDRIVE and OpenSCENARIO formats.
4. Physical Testing: Various scenarios were tested on the AstaZero proving ground using vehicles equipped with advanced sensors and emergency braking systems.
5. Simulations: Partners conducted virtual tests using the respective tool chains. The simulations aimed to replicate physical test conditions and gather comparable data.
6. Data Comparison: Physical and simulated test data were compared to evaluate fidelity levels and trustworthiness. Metrics such as time to collision (TTC), braking distances, and object detection errors were analysed.

Five key case studies were tested:

1. Automated Lane Keeping System (ALKS)
2. Car-to-Car Front Turn-Across-Path (CCFTap)
3. Car in Curve
4. S-Curve
5. Occluded Child

Each scenario focused on different aspects of vehicle dynamics, sensor performance, and emergency braking responses. For instance, the Occluded Child scenario tested automatic emergency braking when a child runs out from behind parked cars.

The project identified gaps between physical and simulated test results, such as differences in braking activations between physical test and simulation. It also highlighted the need for improving simulation tools' ability to replicate real-world vehicle behaviour accurately.

Key findings include:

• Virtual tests can be reliable but require tuning to achieve higher fidelity.
• Physical tests remain crucial for validating simulation models.
• Establishing standardized KPIs for virtual testing is essential to enhance credibility.

The project faced several challenges such as:

• Variability in sensor models across partners.
• Human factors introducing inconsistencies in physical tests.
• Limitations of existing simulation tools to accurately replicate real-world scenarios.

A comprehensive list of challenges was compiled to guide future research and development efforts.

EVIDENT successfully demonstrated the potential of virtual validation for ADAS and AD features. The project contributed to developing methodologies for comparing physical and virtual tests and provided insights into the requirements for credible virtual toolchains.

Future research is recommended to focus on refining simulation validation methods, improving data synchronization methods, and addressing identified challenges to make virtual validation a practical and reliable component of automotive software development.