Publications (new)

We propose SafeMPO, a constrained reinforcement learning framework that incorporates safety constraints directly into the policy optimization process through safety-aware policy updates. By jointly optimizing task performance and constraint satisfaction, SafeMPO improves policy learning while reducing safety violations and maintaining strong overall performance.

We propose the Safe Deep Successor Representation (SafeDSR), a novel method that allows quick retraining of policies towards new cost structures. SafeDSR extends the Deep Successor Representation to Constrained Reinforcement Learning by introducing a single learnable weight matrix to decouple the learned value function across dynamics, rewards, and costs.

We propose a novel objective function for tailoring error correction codes to specific noise structures by maximizing the distinguishability between quantum states after a noise channel, ensuring efficient recovery operations. We formalize this concept with the distinguishability loss function, serving as a machine learning objective to discover resource-efficient encoding circuits optimized for given noise characteristics. We implement this methodology using variational techniques, termed variational quantum error correction. Our approach yields codes with desirable theoretical and practical properties and surpasses standard codes of comparable size under structured noise. We also provide proof-of-concept demonstrations on IBM and IQM hardware devices, highlighting the practical relevance of our procedure.

We propose variational autoencoders (VAEs) for disentanglement to extract essential latent features that enable accurate classification of interferences. We demonstrate that the disentanglement approach can be leveraged for both data compression and data augmentation by interpolating the lower-dimensional latent representations of signal power. To validate our approach, we evaluate three VAE variants—vanilla, factorized, and conditional generative—and benchmark 19 state-of-the-art VAE and generative models on five distinct datasets, including three collected in controlled indoor environments and two real-world highway datasets. In addition, we conduct extensive hyperparameter searches to optimize performance. Our proposed VAE achieves a data compression rate ranging from 512 to 8192 and achieves an accuracy up to 99.92%. Quantizing our model from float32 to int8 results in a fourfold reduction in model weight size.

We present a novel NAS strategy for edge deployment using multi-objective Bayesian optimization (MOBOpt) and reinforcement learning (RL).

We introduce a framework based on ZX calculus, graph-neural networks and reinforcement learning for quantum circuit optimization. By combining reinforcement learning and tree search, our method addresses the challenge of selecting optimal sequences of ZX calculus rewrite rules. Instead of relying on existing heuristic rules for minimizing circuits, our method trains a novel reinforcement learning policy that directly operates on ZX-graphs, therefore allowing us to search through the space of all possible circuit transformations to find a circuit significantly minimizing the number of CNOT gates. This way we can scale beyond hard-coded rules towards discovering arbitrary optimization rules. We demonstrate our method’s competetiveness with state-of-the-art circuit optimizers and generalization capabilities on large sets of diverse random circuits.

We explore variational autoencoders as a way to train unsupervised, i.e., without labels, feature extractors for image classification on a Cortex-M MCU. Using these feature extractors, we then train classification heads using small labeled representative sets of data to solve classification tasks. Our approach significantly reduces the amount of labeled data required, while enabling full DNN training from scratch on resource-constrained devices.

This paper introduces a domain incremental learning (DIL) approach for dynamic 5G indoor localization, called 5GDIL, enabling rapid adaptation to environmental changes. We present a novel similarity-aware sampling technique based on the Chebyshev distance, designed to efficiently select specific exemplars from the previous environment while training only on the modified regions of the new environment. This avoids the need to train on the entire region, significantly reducing the time and resources required for adaptation without compromising localization accuracy. This approach requires as few as 50 exemplars from adaptation domains, significantly reducing training time while maintaining high positioning accuracy in previous environments. Comparative evaluations against state-of-the-art DIL techniques on a challenging real-world indoor dataset demonstrate the effectiveness of the proposed sample selection method. Our approach is adaptable to real-world non-line-ofsight propagation scenarios and achieves an MAE positioning error of 0.261 meters, even under dynamic environmental conditions.

Code: https://gitlab.cc-asp.fraunhofer.de/5g-pos/5g-dil

We propose an approach for detecting and classifying interference while estimating the distance, azimuth, and elevation of jamming sources. We evaluate 128 vision encoder and time-series models to identify the highest-performing methods for each task and introduce an attention-based fusion framework that integrates in-phase and quadrature (IQ) samples with Fast Fourier Transform (FFT)-computed spectrograms while incorporating 22 AoA features
to enhance localization accuracy.

This paper proposes a method to learn an effective representation between previous and newly encountered class prototypes. We propose a prototypical network with a Bayesian learning-driven contrastive loss (BLCL), tailored specifically for class-incremental learning scenarios. We introduce a contrastive loss that incorporates novel classes into the latent representation by reducing intra-class and increasing inter-class distance. Our approach dynamically adapts the balance between the cross-entropy and contrastive loss functions with a Bayesian learning technique.

To enable valid performance comparisons and to streamline current research in this area, we propose a novel benchmarking methodology, which is based on a statistical estimator for sample complexity and a definition of statistical outperformance. Furthermore, considering QRL, our methodology casts doubt on some previous claims regarding its superiority. We conducted experiments on a novel benchmarking environment with flexible levels of complexity. While we still identify possible advantages, our findings are more nuanced overall. We discuss the potential limitations of these results and explore their implications for empirical research on quantum advantage in QRL.

We propose a novel NAS strategy for TinyML based on multi-objective Bayesian optimization (MOBOpt) and an ensemble of competing parametric policies trained using augmented random search (ARS) reinforcement learning (RL) agents. Our methodology aims at efficiently finding tradeoffs between a DNN’s predictive accuracy, memory requirements on a given target system, and computational complexity. Our experiments show that we consistently outperform existing MOBOpt approaches on different datasets and architectures such as ResNet-18 and MobileNetv3.

A framework of six orthogonal metrics for gradient- or perturbation-based post-hoc visual interpretation methods. Its designed for time series classification and segmentation tasks.

We introduce an extensive dataset comprising snapshots obtained from a low-frequency antenna that capture various generated interferences within a large-scale environment, including controlled multipath effects. Our objective is to assess the resilience of machine learning (ML) models against environmental changes, such as multipath effects, variations in interference attributes, such as interference class, bandwidth, and signal power, the accuracy of jamming device localization, and the constraints imposed by snapshot input lengths. Furthermore, we evaluate the performance of a diverse set of 129 distinct vision encoder models across all tasks.

We propose an FL approach using few-shot learning and aggregation of the model weights on a global server. We introduce a dynamic early stopping method to balance out-of-distribution classes based on representation learning, specifically utilizing the maximum mean discrepancy of feature embeddings between local and global models. An exemplary application of FL is to orchestrate machine learning models along highways for interference classification based on snapshots from global navigation satellite system (GNSS) receivers.

Dataset/Code: https://gitlab.cc-asp.fraunhofer.de/darcy_gnss/federated_learning

Fingerprint-based passive localization enables high localization accuracy using low-cost UWB IoT radio sensors. However, fingerprinting demands extensive effort for data acquisition. The concept of channel charting reduces this effort by modeling and projecting the manifold of channel state information (CSI) onto a 2D coordinate space. So far, researchers have only applied this concept to active radio localization, where a mobile device intentionally and actively emits a specific signal.In this paper, we apply channel charting to passive localization. We use a pedestrian dead reckoning (PDR) system to estimate a target’s velocity and derive a distance matrix from it. We then use this matrix to learn a distance-preserving embedding in 2D space, which serves as a fingerprinting model. In our experiments, we deploy six nodes in a fully connected ultra-wideband (UWB) mesh network to show that our method achieves high localization accuracy, with an average error of just 0.24 m, even when we train and test on different targets.

We show a channel charting architecture that can natively handle variable numbers of CSI measurements and arbitrary antenna subsets. Across two public real-world SISO and MIMO datasets, AdaPos maintains state-of-the-art localization accuracy under missing-antenna conditions, replaces roughly 57 configuration-specific models with a single unified model, and provides resilience to both individual antenna failures and full-array outages.

We provide an overview of AI/ML-related discussions in 3GPP standardization, highlighting key use cases, architectural considerations, and technical requirements. We outline open research challenges and opportunities where academic and industrial communities can contribute to shaping the future of AI-enabled wireless systems.

We introduce Sparse pretrained Radio Transformer (SpaRTran), a hybrid method based on the concept of compressed sensing for wireless channels. SpaRTran builds around a wireless channel model that constrains the optimization procedure to physically meaningful solutions and induces a strong inductive bias, cutting positioning error by up to 28% and increasing top-1 codebook selection accuracy for beamforming by 26 percentage points.

Quantum error correction is critical for protecting quantum computations from noise, yet standard protocols demand significant qubit overhead. To address this, we introduce a learning-based approach that allows tailoring encodings to specific noise channels. This approach is based on a novel loss function that maximizes the distinguishability of noisy quantum states. We thereby realize a variational quantum algorithm that discovers new resource-efficient codes, which empirically outperform standard quantum error correction schemes on various noise structures. We furthermore derive promising theoretical properties of these variational quantum error correction (VarQEC) codes and perform proof-of-concept experiments on quantum hardware.

In this paper, we investigate this potential advantage by proposing a batch RL algorithm that utilizes variational quantum circuits (VQCs) as function approximators within the discrete batch-constraint deep Q-learning (BCQ) algorithm. Additionally, we introduce a novel data re-uploading scheme by cyclically shifting the order of input variables in the data encoding layers. We evaluate the efficiency of our algorithm on the OpenAI CartPole environment and compare its performance to the classical neural network-based discrete BCQ.

The Constrained Markov Decision Process (CMDP) formulation allows to solve safety-critical decision making tasks that are subject to constraints. While CMDPs have been extensively studied in the Reinforcement Learning literature, little attention has been given to sampling-based planning algorithms such as MCTS for solving them. Previous approaches perform conservatively with respect to costs as they avoid constraint violations by using Monte Carlo cost estimates that suffer from high variance. We propose Constrained MCTS (C-MCTS), which estimates cost using a safety critic that is trained with Temporal Difference learning in an offline phase prior to agent deployment. The critic limits exploration by pruning unsafe trajectories within MCTS during deployment. C-MCTS satisfies cost constraints but operates closer to the constraint boundary, achieving higher rewards than previous work. As a nice byproduct, the planner is more efficient w.r.t. planning steps. Most importantly, under model mismatch between the planner and the real world, C-MCTS is less susceptible to cost violations than previous work.

Linear systems of equations can be found in various mathematical domains, as well as in the field of machine learning. By employing noisy intermediate-scale quantum devices, variational solvers promise to accelerate finding solutions for large systems. Although there is a wealth of theoretical research on these algorithms, only fragmentary implementations exist. To fill this gap, we have developed the variational-lse-solver framework, which realizes existing approaches in literature, and introduces several enhancements. The user-friendly interface is designed for researchers that work at the abstraction level of identifying and developing end-to-end applications.

We implicitly estimate the number of MPCs and achieve subsample accuracy without using computationally intensive super-resolution techniques. Our approach outperforms known methods in detection performance and accuracy at different bandwidths. Our ablation study shows exceptional results on simulated and real datasets and generalizes to unknown radio environments.

We propose a few-shot learning (FSL) approach to adapt to new interference classes. Our method employs quadruplet selection for the model to learn representations using various positive and negative interference classes. Furthermore, our quadruplet vari-ant selects pairs based on the aleatoric and epistemic uncertainty to differentiate between similar classes. We recorded a dataset at a motorway with eight interference classes on which our FSL method with quadruplet loss outperforms other FSL techniques in jammer classification accuracy with 97.66%. https://gitlab.cc-asp.fraunhofer.de/darcy_gnsslFIOT_highway.