Automated Driving Systems (ADS) employ various techniques for operation perception, task planning and vehicle control. For driving on public roads, it is critical to guarantee the operational safety of such systems by attaining Minimal Risk Condition (MRC) despite unexpected environmental disruptions, human errors, functional faults and security attacks. This paper proposes a methodology to automatically identify potentially highly critical operational conditions by leveraging the design-time information in terms of vehicle architecture models and environment models. To identify the critical operating conditions, these design-time models are combined systematically with a variety of faults models for revealing the system behaviours in the presence of anomalies. The contributions of this paper are summarized as follows: 1) The design of a method for extracting related internal and external operational conditions from different system models. 2) The design of software services for identifying critical parameters and synthesizing operational data with fault injection. 3) The design for supporting operation simulation and data analysis.

Artificial Intelligence (AI) techniques through Learning-Enabled Components (LEC) are widely employed in Automated Driving Systems (ADS) to support operation perception and other driving tasks relating to planning and control. Therefore, the risk management plays a critical role in assuring the operational safety of ADS. However, the probabilistic and nondeterministic nature of LEC challenges the safety analysis. Especially, the impacts of their functional faults and incompatible external conditions are often difficult to identify. To address this issue, this article presents a simulation-aided approach as follows: 1) A simulation-aided operational data generation service with the operational parameters extracted from the corresponding system models and specifications; 2) A Fault Injection (FI) service aimed at high-dimensional sensor data to evaluate the robustness and residual risks of LEC. 3) A Variational Bayesian (VB) method for encoding the collected operational data and supporting an effective estimation of the likelihood of operational conditions. As a case study, the paper presents the results of one experiment, where the behaviour of an Autonomous Emergency Braking (AEB) system is simulated under various weather conditions based on the CARLA driving simulator. A set of fault types of cameras, including solid occlusion, water drop, salt and pepper, are modelled and injected into the perception module of the AEB system in different weather conditions. The results indicate that our framework enables to identify the critical faults under various operational conditions. To approximate the critical faults in undefined weather, we also propose Variational Autoencoder (VAE) to encode the pixel-level data and estimate the likelihood.

Flexible wearable sensor electronics, combined with advanced software functions, pave the way toward increasingly intelligent healthcare devices. One important application area is limb prosthesis, where printed flexible sensor solutions enable efficient monitoring and assessing of the actual intra-socket dynamic operation conditions in clinical and other more natural environments. However, the data collected by such sensors suffer from variations and errors, leading to difficulty in perceiving the actual operational conditions. This paper proposes a novel method for detecting anomalies in the data that are collected for measuring the intra-socket dynamic operation conditions by printed flexible wearable sensors. A discrete generative model based on Variational AutoEncoder (VAE) is used first to encode the collected multi-variant time-series data in terms of latent states. After that, a clustering method based on the Self-Organizing Map (SOM) is used to acquire discrete and interpretable representations of the VAE encoded latent states. An adaptive Markov chain is utilized to detect anomalies by quantifying state transitions and revealing temporal dependencies. The contributions of the proposed architecture conclude as follows: (1) Using the VAE-SOM hybrid model to regularize the continues data as discrete states, supporting interpreting the operational data to analytic models. (2) Employing adaptive Markov chains to generalize the transitions of these states, allowing to model the complex operational conditions. Compared with benchmark methods, our architecture is validated via two public datasets and achieves the best F1 scores. Moreover, we measure the run-time performance of this lightweight architecture. The results indicate that the proposed method performs low computational complexity, facilitating the applications on real-life productions.

Anomaly detection is essential in Industrial Cyber-Physical Systems (ICPS) for monitoring both system and environmental conditions. However, effective anomaly detection remains a continuous challenge due to the complexity and diversity of operational features in both spatial and temporal domains of such systems. Current data-driven approaches utilize Artificial Intelligence (AI)-enabled methods, such as Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN), to process and analyze the collected operational data. From the industrial perspective, a key challenge lies in enhancing the effectiveness and explainability, as these methods often exhibit opaque and sophisticated network structures when dealing with complex operational data. Meanwhile, recent advancements in Graph Neural Networks (GNN) demonstrate its effectiveness of supporting the analysis of complex relationships and dependencies. The adoption of GNN poses however inherent challenges when the data are unstructured. This paper designs a novel framework that integrates Self-Organizing Maps (SOM) and GNN for the analysis of complex operational relationships and dependencies. In particular, the adoption of SOM allows the generation graph-structured data from unstructured raw datasets and thereby enhances a GNN’s ability to differentiate normal and anomalous conditions effectively. As case studies, we select multiple public datasets to compare the performance of the proposed framework with other benchmark methods. The results show that the proposed methods present promising results. Additionally, compared to other baseline methods that use GNN-based structures to detect anomalies, the proposed framework achieves the highest F1-Score.

Human Activity Recognition (HAR) refers to a field that aims to identify human activitiesby adopting multiple techniques. In this field, different applications, such as smart homes andassistive robots, are introduced to support individuals in their Activities of Daily Living (ADL)by analyzing data collected from various sensors. Apart from wearable sensors, the adoption ofcamera frames to analyze and classify ADL has emerged as a promising trend for achieving theidentification and classification of ADL. To accomplish this, the existing approaches typically rely onobject classification with pose estimation using the image frames collected from cameras. Given theexistence of inherent correlations between human–object interactions and ADL, further efforts areoften needed to leverage these correlations for more effective and well justified decisions. To this end,this work proposes a framework where Graph Neural Networks (GNN) are adopted to explicitlyanalyze human–object interactions for more effectively recognizing daily activities. By automaticallyencoding the correlations among various interactions detected through some collected relational data,the framework infers the existence of different activities alongside their corresponding environmentalobjects. As a case study, we use the Toyota Smart Home dataset to evaluate the proposed framework.Compared with conventional feed-forward neural networks, the results demonstrate significantlysuperior performance in identifying ADL, allowing for the classification of different daily activitieswith an accuracy of 0.88. Furthermore, the incorporation of encoded information from relational dataenhances object-inference performance compared to the GNN without joint prediction, increasingaccuracy from 0.71 to 0.77.