This paper addresses the challenge of designing resilient control systems under Denial of Service (DoS) attacks for discrete-Time systems. A learning-based framework is proposed to reconstruct lost measurements and compute optimal controllers using input-output data, eliminating the need for a complete system model. By leveraging state reconstruction techniques, the framework estimates missing information during DoS periods, ensuring robust control performance. Two algorithms, based on policy iteration (PI) and value iteration (VI), are developed to learn the optimal feedback control policy. The effectiveness of the proposed methodology is illustrated via a numerical example. © 2025 The Authors.

This paper considers the constraint-satisfaction control problem for a class of linear systems subject to multiple output constraints. A novel output-feedback control framework is developed, with a refined quadratic programming (QP) based virtual controller and a Luenberger observer. The main contribution lies in the refined QP-based virtual controller, which employs a positive basis to construct its feasible set, ensuring (local) Lipschitz continuity while preserving robustness. Under the assumptions that the plant is controllable and observable and that the constraints satisfy a disjointness condition, the proposed controller can guarantee the satisfaction of all output constraints. The effectiveness of the proposed controller is demonstrated through numerical simulations on an identified quadrotor model. © 2025 IEEE.

This paper studies the safety-critical control of a class of underactuated systems, with a focus on a wheeled inverted pendulum as a case study. A safety-critical controller is desired to simultaneously keep the pendulum upright, restrict the position of the wheel within a desired range, and ensure that the rotational angular velocity of the wheel tracks a nominal reference signal as closely as possible. We first reformulate the safety constraints as saturated constraints on the state variables of a cascade connection of an integrator and a feedforward system, thereby mitigating conflicts among the safety constraints. Then, to ensure the satisfaction of the constraints, we propose a safety-critical controller that consists of a safety-critical virtual controller based on control barrier functions (CBFs) and a tracking controller designed using forwarding methods. Nonlinear small-gain techniques are employed to ensure the stable integration of the two controllers and the achievement of the safety-critical control objectives. © 2025 EUCA.

This paper focuses on third-order multi-agent systems with directed communication networks. It is for the first time that the finite-time consensus problem is solved for third-order multi-agent systems under digraphs containing spanning trees. Two explicit-coefficient finite-time protocols, namely, edge-based and node-based, are proposed. First, we prove that asymptotic consensus is achievable with the reduced linear protocol. Then, we show that finite-time consensus is achievable with our explicit-coefficient finite-time protocols through homogeneous theory. Finally, numerical simulations are carried out to demonstrate the effectiveness of the proposed protocols. © 2024 IEEE.

This paper investigates the distributed feedback optimization problem of nonlinear multi-agent systems. In such systems, each agent can measure the relative outputs between itself and its neighbors but lacks access to their absolute states and internal controller states. By combining distributed optimization and singular perturbation methods, a novel distributed controller design is presented, that relies solely on each agent's real-time gradient values of its local objective function and its relative output measurements to neighboring agents. The boundedness of the closed-loop signals and the convergence of the agent outputs to the minimizer of the total cost are proved rigorously. A numerical example is conducted to validate the effectiveness of the proposed approach. © 2024 EUCA.

In this paper, we have proposed a resilient reinforcement learning method for discrete-time linear systems with unknown parameters, under denial-of-service (DoS) attacks. The proposed method is based on policy iteration that learns the optimal controller from input-state data amidst DoS attacks. We achieve an upper bound for the DoS duration to ensure closed-loop stability. The resilience of the closed-loop system, when subjected to DoS attacks with the learned controller and an internal model, has been thoroughly examined. The effectiveness of the proposed methodology is demonstrated on an inverted pendulum on a cart. © 2024 IEEE.

Risk sensitivity is a fundamental aspect of biological motor control that accounts for both the expectation and variability of movement cost in the face of uncertainty. However, most computational models of biological motor control rely on model-based risk-sensitive optimal control, which requires an accurate internal representation in the central neural system to predict the outcomes of motor commands. In reality, the dynamics of human-environment interaction is too complex to be accurately modeled, and noise further complicates system identification. To address this issue, this paper proposes a novel risk-sensitive computational mechanism for biological motor control based on reinforcement learning (RL) and adaptive dynamic programming (ADP). The proposed ADP-based mechanism suggests that humans can directly learn an approximation of the risk-sensitive optimal feedback controller from noisy sensory data without the need for system identification. Numerical validation of the proposed mechanism is conducted on the arm-reaching task under divergent force field. The preliminary computational results align with the experimental observations from the past literature of computational neuroscience. © 2023 IEEE.

In this paper, we propose a robust reinforcement learning method for a class of linear discrete-time systems to handle model mismatches that may be induced by sim-to-real gap. Under the formulation of risk-sensitive linear quadratic Gaussian control, a dual-loop policy optimization algorithm is proposed to iteratively approximate the robust and optimal controller. The convergence and robustness of the dual-loop policy optimization algorithm are rigorously analyzed. It is shown that the dual-loop policy optimization algorithm uniformly converges to the optimal solution. In addition, by invoking the concept of small-disturbance input-to-state stability, it is guaranteed that the dual-loop policy optimization algorithm still converges to a neighborhood of the optimal solution when the algorithm is subject to a sufficiently small disturbance at each step. When the system matrices are unknown, a learning-based off-policy policy optimization algorithm is proposed for the same class of linear systems with additive Gaussian noise. The numerical simulation is implemented to demonstrate the efficacy of the proposed algorithm. © 2023 L. Cui, T. Başar & Z.-P. Jiang.

In this paper, we solve the optimal output regulation problem for discrete-time systems without precise knowledge of the system model. Drawing inspiration from reinforcement learning and adaptive dynamic programming, a data-driven solution is developed that enables asymptotic tracking and disturbance rejection. Notably, it is discovered that the proposed approach for discrete-time output regulation differs from the continuous-time approach in terms of the persistent excitation condition required for policy iteration to be unique and convergent. To address this issue, a new persistent excitation condition is introduced to ensure both uniqueness and convergence of the data-driven policy iteration. The efficacy of the proposed methodology is validated by an inverted pendulum on a cart example. © 2023 IEEE.