Collaborative Research:NewSpectrum:Toward Untethered Extended Reality Through Wireless Sensing and Communications Co-design

项目来源

美国国家科学基金(NSF)

项目主持人

Shiwen Mao

项目受资助机构

Auburn University

项目编号

2434053

财政年度

2025,2024

立项时间

未公开

研究期限

未知 / 未知

项目级别

国家级

受资助金额

340000.00美元

学科

未公开

学科代码

未公开

基金类别

Standard Grant

关键词

SWIFT-Spectrum Innov Futr Tech ; EARS ; Wireless comm&sig processing ; EDUCATION AND WORKFORCE ; EXP PROG TO STIM COMP RES

参与者

未公开

参与机构

AUBURN UNIVERSITY

项目标书摘要:Extended reality(XR)offers users immersive experience in virtual worlds,and enables a broad range of applications(i.e.,training,gaming,and medical imaging).There has been an increasing interest on the study of the deployment of XR services over next era of wireless networks(nextE),so as to provide seamless wireless connectivity for XR users to eliminate the wired connection constraints thus enabling future wireless devices to use VR services.However,the few prior studies have two major limitations:1)They are mainly focused on network optimization for XR data transmission and are lacking in novel user behavior sensing methods,2)Their XR sensing methods mostly rely on statically installed sensors or cameras,which also restrict the operation range of users and suffer from user movement and blockage,3)they are restricted to either a single XR system,or multiple XR systems where each XR system consists of only one user and hence cannot be applied for multi-user XR systems.To address the aforementioned challenges,a holistic wireless XR framework is developed,which utilizes mmWave for joint XR user movement detection and XR data transmission while satisfying the joint communication,computing,sensing,and XR service requirements.If successful,this project will enable highly efficient and robust wireless enabled XR networks and applications,with significantly enhanced accuracy,resilience,and user experience.The project integrates the research insights into new modules for communication and network related courses and hosts outreach activities with the vision of advancing the participation of underrepresented minorities in STEM fields.The untethered XR project presents a cutting-edge solution for eliminating XR wired connections and limitations of XR user activity space by utilizing mmWave,machine learning,edge computing,and joint sensing and communications technologies to truly unleashing the high potential of XR via:1)developing novel mmWave-based sensing methods which exploit complex valued channel state information and radio map information to detect the full-body movements of multiple XR users;2)designing a novel collaborative reinforcement learning(RL)framework to produce a low-complexity and reliable collaborative learning process that enables distributed XR access points(APs)to jointly optimize XR sensing and data transmission in order to improve the quality-of-experience of XR users;3)building an open-source software platform and hardware testbed to validate the wireless XR solutions.This project provides a rich environment and virtualized platform that facilitate educating and training students at multiple levels.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人员信息

Shiwen Mao(Principal Investigator):smao@auburn.edu;

机构信息

【Auburn University(Performance Institution)】StreetAddress:321-A INGRAM HALL,AUBURN,Alabama,United States/ZipCode:368490001;【AUBURN UNIVERSITY】StreetAddress:321-A INGRAM HALL,AUBURN,Alabama,United States/PhoneNumber:3348444438/ZipCode:36849;

项目主管部门

Directorate for Engineering(ENG)-Division of Electrical,Communications and Cyber Systems(ECCS)

项目官员

Huaiyu Dai(Email:hdai@nsf.gov;Phone:7032924568)

  • 排序方式:
  • 1
  • /
  • 1.Generative AI Enabled Robust Data Augmentation for Wireless Sensing in ISAC Networks

    • 关键词:
    • Sensors; Training; Diffusion models; Data models; Data augmentation;Feature extraction; Reliability; Location awareness; Wireless sensornetworks; Wireless communication; Integrated sensing and communications;generative AI; data augmentation;JOINT RADAR
    • Wang, Jiacheng;Zhao, Changyuan;Du, Hongyang;Sun, Geng;Kang, Jiawen;Mao, Shiwen;Niyato, Dusit;In Kim, Dong
    • 《IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS》
    • 2026年
    • 44卷
    • 期刊

    Integrated sensing and communication (ISAC) uses the same software and hardware resources to achieve both communication and sensing functionalities. Thus, it stands as one of the core technologies of 6G and has garnered significant attention in recent years. In ISAC systems, a variety of machine learning models are trained to analyze and identify signal patterns, thereby ensuring reliable sensing and communications. However, considering factors such as communication rates, costs, and privacy, collecting sufficient training data from various ISAC scenarios for these models is impractical. Hence, this paper introduces a generative AI (GenAI) enabled robust data augmentation scheme. The scheme first employs a conditioned diffusion model trained on a limited amount of collected CSI data to generate new samples, thereby enhancing the sample quantity. Building on this, the scheme further utilizes another diffusion model to enhance the sample quality, thereby facilitating the data augmentation in scenarios where the original sensing data is insufficient and unevenly distributed. Moreover, we propose a novel algorithm to estimate the acceleration and jerk of signal propagation path length changes from CSI. We then use the proposed scheme to enhance the estimated parameters and detect the number of targets based on the enhanced data. The evaluation reveals that our scheme improves the detection performance by up to 70%, demonstrating reliability and robustness, which supports the deployment and practical use of the ISAC network.

    ...
  • 2.Unified Packet Compression and Model Adaptation for Integrated Sensing and Multi-Modal Communications

    • 关键词:
    • Data models; Predictive models; Computational modeling; Adaptationmodels; Transformers; Entropy; Data compression; Redundancy; Imagecoding; Data communication; Byte-based model; Transformer; packetcompression; multi-modal sensing and communications;WIRELESS SENSOR NETWORKS; PREDICTION; EFFICIENCY; REDUCTION
    • Luo, Xuanhao;Li, Zhouyu;Chen, Mingzhe;Yu, Ruozhou;Mao, Shiwen;Liu, Yuchen
    • 《IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS》
    • 2026年
    • 44卷
    • 期刊

    Integrated sensing and communication systems face critical challenges, including limited bandwidth, power constraints, and varying communication conditions, which demand efficient data transmission and processing strategies. This paper introduces, ByteTrans, a novel joint optimization framework that integrates byte-level predictive modeling with adaptive model scheduling to maximize data transmission efficiency while adhering to communication and computational constraints. The proposed framework employs Transformer-based models to predict and compress data packets losslessly, leveraging the inherent redundancy in multi-modal network data. Such a unified data compression approach predicts occurring byte probabilities, encodes them as ranks using lossless entropy coding, and efficiently reduces data size and entropy across diverse modalities. Then, a dynamic adaptation strategy selects the optimal compression model based on packet characteristics and channel conditions, ensuring efficient operation across heterogeneous sensor environments. Experimental results validate that our scheme achieves compression rates exceeding 50%, while showcasing substantial reductions in communication time and bandwidth usage under both normal and adverse channel conditions. Furthermore, we effectively implement these models across various real-world edge sensors and servers, showcasing their practicality and efficiency in various network applications. By addressing the trade-offs between achieving lower compression ratios and limiting computational and energy consumption, this work establishes a scalable and robust solution for data management in multi-modal communication systems.

    ...
  • 3.Optimizing Communication and Device Clustering for Clustered Federated Learning With Differential Privacy

    • 关键词:
    • Training; Resource management; Servers; Computational modeling;Optimization; Noise; Convergence; Clustering algorithms; Federatedlearning; Data models; And resource allocation; clustered federatedlearning; differential privacy; multi-agent reinforcement learning
    • Wei, Dongyu;Xu, Xiaoren;Mao, Shiwen;Chen, Mingzhe
    • 《IEEE TRANSACTIONS ON MOBILE COMPUTING》
    • 2026年
    • 25卷
    • 1期
    • 期刊

    In this paper, a secure and communication-efficient clustered federated learning (CFL) design is proposed. In our model, several base stations (BSs) with heterogeneous task-handling capabilities and multiple users with non-independent and identically distributed (non-IID) data jointly perform CFL training incorporating differential privacy (DP) techniques. Since each BS can process only a subset of the learning tasks and has limited wireless resource blocks (RBs) to allocate to users for federated learning (FL) model parameter transmission, it is necessary to jointly optimize RB allocation and user scheduling for CFL performance optimization. Meanwhile, our considered CFL method requires devices to use their limited data and FL model information to determine their task identities, which may introduce additional communication overhead. We formulate an optimization problem whose goal is to minimize the training loss of all learning tasks while considering device clustering, RB allocation, DP noise, and FL model transmission delay. To solve the problem, we propose a novel dynamic penalty function assisted value decomposed multi-agent reinforcement learning (DPVD-MARL) algorithm that enables distributed BSs to independently determine their connected users, RBs, and DP noise of the connected users but jointly minimize the training loss of all learning tasks across all BSs. Different from the existing MARL methods that assign a large penalty for infeasible actions, we propose a novel penalty assignment scheme that assigns penalty depending on the number of devices that cannot meet communication constraints (e.g., delay), which can guide the MARL scheme to quickly find valid actions, thus improving the convergence speed. Simulation results show that the DPVD-MARL can improve the convergence rate by up to 20% and the ultimate accumulated rewards by 15% compared to independent Q-learning.

    ...
  • 4.Contextual Combinatorial Beam Management via Online Probing for Multiple Access mmWave Wireless Networks

    • 关键词:
    • Transceivers; Millimeter wave communication; Wireless networks; Arraysignal processing; Resource management; Next generation networking;NOMA; Heuristic algorithms; Vehicle dynamics; Machine learningalgorithms; Beam management; mmWave; transceiver pairing; wirelessnetworks; multi-armed bandit; contextual awareness;ALIGNMENT
    • Li, Zhizhen;Luo, Xuanhao;Chen, Mingzhe;Xu, Chenhan;Mao, Shiwen;Liu, Yuchen
    • 《IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS》
    • 2025年
    • 43卷
    • 3期
    • 期刊

    Due to the exponential increase in wireless devices and a diversification of network services, unprecedented challenges, such as managing heterogeneous data traffic and massive access demands, have arisen in next-generation wireless networks. To address these challenges, there is a pressing need for the evolution of multiple access schemes with advanced transceivers. Millimeter-wave (mmWave) communication emerges as a promising solution by offering substantial bandwidth and accommodating massive connectivities. Nevertheless, the inherent signaling directionality and susceptibility to blockages pose significant challenges for deploying multiple transceivers with narrow antenna beams. Consequently, beam management becomes imperative for practical network implementations to identify and track the optimal transceiver beam pairs, ensuring maximum received power and maintaining high-quality access service. In this context, we propose a Contextual Combinatorial Beam Management (CCBM) framework tailored for mmWave wireless networks. By leveraging advanced online probing techniques and integrating predicted contextual information, such as dynamic link qualities in spatial-temporal domain, CCBM aims to jointly optimize transceiver pairing and beam selection while balancing the network load. This approach not only facilitates multiple access effectively but also enhances bandwidth utilization and reduces computational overheads for real-time applications. Theoretical analysis establishes the asymptotically optimality of the proposed approach, complemented by extensive evaluation results showcasing the superiority of our framework over other state-of-the-art schemes in multiple dimensions.

    ...
  • 5.Generative AI Based Secure Wireless Sensing for ISAC Networks

    • 关键词:
    • Sensors; Wireless communication; Diffusion models; Wireless sensornetworks; Communication system security; Performance evaluation;Security; Integrated sensing and communication; Fluctuations; Accuracy;Generative AI; integrated sensing and communication; wireless sensingsecurity
    • Wang, Jiacheng;Du, Hongyang;Liu, Yinqiu;Sun, Geng;Niyato, Dusit;Mao, Shiwen;In Kim, Dong;Shen, Xuemin
    • 《IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY》
    • 2025年
    • 20卷
    • 期刊

    Integrated sensing and communications (ISAC) is one of the crucial technologies for 6G, and channel state information (CSI) based sensing serves as an essential part of ISAC. However, current research on ISAC focuses mainly on improving sensing performance, overlooking security issues, particularly the unauthorized sensing of users. Hence, this paper proposes a diffusion model based secure sensing system (DFSS). Specifically, we first propose a discrete conditional diffusion model to generate graphs with nodes and edges, which guides the ISAC system to appropriately activate wireless links and nodes, ensuring the sensing performance while minimizing the operation cost. Using the activated links and nodes, DFSS then employs the continuous conditional diffusion model to generate safeguarding signals, which are next modulated onto the pilot at the transmitter to mask fluctuations caused by user activities. As such, only authorized ISAC devices with the safeguarding signals can extract the true CSI for sensing, while unauthorized devices are unable to perform the effective sensing. Experiment results demonstrate that DFSS can reduce the activity recognition accuracy of the unauthorized devices by approximately 70%, effectively shield the user from the illegitimate surveillance.

    ...
  • 6.Generative AI Based Secure Wireless Sensing for ISAC Networks

    • 关键词:
    • Sensors; Wireless communication; Diffusion models; Wireless sensornetworks; Communication system security; Performance evaluation;Security; Integrated sensing and communication; Fluctuations; Accuracy;Generative AI; integrated sensing and communication; wireless sensingsecurity
    • Wang, Jiacheng;Du, Hongyang;Liu, Yinqiu;Sun, Geng;Niyato, Dusit;Mao, Shiwen;In Kim, Dong;Shen, Xuemin
    • 《IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY》
    • 2025年
    • 20卷
    • 期刊

    Integrated sensing and communications (ISAC) is one of the crucial technologies for 6G, and channel state information (CSI) based sensing serves as an essential part of ISAC. However, current research on ISAC focuses mainly on improving sensing performance, overlooking security issues, particularly the unauthorized sensing of users. Hence, this paper proposes a diffusion model based secure sensing system (DFSS). Specifically, we first propose a discrete conditional diffusion model to generate graphs with nodes and edges, which guides the ISAC system to appropriately activate wireless links and nodes, ensuring the sensing performance while minimizing the operation cost. Using the activated links and nodes, DFSS then employs the continuous conditional diffusion model to generate safeguarding signals, which are next modulated onto the pilot at the transmitter to mask fluctuations caused by user activities. As such, only authorized ISAC devices with the safeguarding signals can extract the true CSI for sensing, while unauthorized devices are unable to perform the effective sensing. Experiment results demonstrate that DFSS can reduce the activity recognition accuracy of the unauthorized devices by approximately 70%, effectively shield the user from the illegitimate surveillance.

    ...
  • 排序方式:
  • 1
  • /