NRI/Collaborative Research:Robotic Disassembly of High-Precision Electronic Devices
项目来源
项目主持人
项目受资助机构
财政年度
立项时间
项目编号
项目级别
研究期限
受资助金额
学科
学科代码
基金类别
关键词
参与者
参与机构
人员信息
机构信息
项目主管部门
项目官员
1.Application-driven multi-modal depth completion in fringe projection profilometry
- 关键词:
- Automation;Electronic Waste;Hard disk storage;Mean square error;Mirrors;Optical depth;Optical projectors;Optical properties;Optical testing;Projection systems ;Recycling;Surface testing;Deep learning;Depth completion;Depthmap;Fringe projection profilometry;Industrial metrology;Modal models;Multi-modal;Multi-modal model;Robotic automation;Synthetic data
- Balasubramaniam, Badrinath;Suresh, Vignesh;Cheng, Yang;Li, Jiaqiong;Li, Beiwen
- 《Optics and Lasers in Engineering》
- 2026年
- 200卷
- 期
- 期刊
Fringe projection profilometry (FPP), while capable of sub-millimeter accuracy at kilohertz speeds, produces sparse and incomplete depth maps when scanning objects with complex, heterogeneous material properties including specular metallic surfaces, mirror-like reflective regions, and absorptive materials. This is due to measurement failures predominantly in mirror-like reflective regions and underexposed areas where fringe patterns are unreliable or absent. Hard disk drives represent a particularly challenging test case for these limitations, exhibiting all of these problematic surface characteristics within a single assembly. Accurate 3D sensing of such components is critical for automated robotic disassembly in e-waste recycling, where valuable materials such as palladium, aluminum, and the rare earth metal neodymium remain largely unrecovered due to lack of recycling infrastructure. Recent zero-shot depth estimation models, while inaccurate for fine-scale, millimeter-level depth prediction, capture useful geometric priors. In this research, we present a multi-modal fusion approach that combines three data sources: sparse depth map computed from FPP, projector-illuminated grayscale image, and the relative depth map from the Depth Anything v2 Foundation Model. Our lightweight fusion network exploits the lower domain gap in geometric features compared to appearance features, enabling effective learning and sim-to-real transfer with limited synthetic and real-world training data. The network learns to predict dense depth in regions where FPP fails, which is then fused with the original sparse measurements to produce complete depth maps. We demonstrate that this approach achieves a mean absolute error and root mean square error of less than 2 mm on both synthetic and real-world test cases, and critically, achieves good reconstruction fidelity in the sparse regions, paving the way for fine-scale robotic disassembly while avoiding the need for extensive surface treatment or large-scale real-world data collection. Furthermore, our approach addresses the primary limitations of FPP on mirror-like reflective surfaces and underexposed regions within a single scan, and demonstrates a potential roadmap for industrial metrology of parts with similarly challenging optical properties. The code for our multi-modal depth completion network, MMDC-Net, will be publicly available at https://github.com/badri999/MMDC-Net © 2025 The Author(s)
...2.Encrypted fringe projection profilometry
- Cheng, Yang;Gui, Zichen;Guan, Le;Jin, Ran;Li, Beiwen
- 《OPTICS LETTERS》
- 2026年
- 51卷
- 4期
- 期刊
We propose encrypted fringe projection profilometry (E-FPP), a co-keyed phase-code framework that embeds encryption directly into the measurement process of fringe projection profilometry (FPP). Two random phase fields and orthogonal temporal codes jointly encode the projected sequence, binding phase retrieval to legitimate key pairs. The method preserves the standard three-step FPP operation while enabling key-dependent 3D reconstruction. Experiments verify accurate authorized reconstruction and strong resistance to single-and multi-frame inference, providing a privacy-preserving solution for optical metrology. (c) 2026 Optica Publishing Group. All rights, including for text and data mining (TDM), Artificial Intelligence (AI) training, and similar technologies, are reserved.
...3.Precision 3D profilometry of consumer-grade computer enclosures using high dynamic range fringe projection
- Zhao, Haiyang;Liu, Chang;Balasubramaniam, Badrinath;Li, Jiaqiong;Song, Jiurun;Liang, Xiao;Zheng, Minghui;Li, Beiwen
- 《APPLIED OPTICS》
- 2025年
- 64卷
- 30期
- 期刊
In recent years, the concentration of precious metals and hazardous pollutants in discarded consumer-grade computer enclosures has increased significantly, coinciding with e-waste generation in Asia reaching approximately 30 million tons annually. However, the high cost and low efficiency of manual disassembly present substantial obstacles to the effective recycling of such enclosures. Robotic disassembly has emerged as a promising alternative. To enable accurate acquisition of three-dimensional (3D) geometric data for robotic operations, we propose a 3D measurement method based on multi-color high dynamic range imaging. This method employs a seven-color illumination strategy and exploits the spectral response characteristics of a color camera to different wavelengths, effectively mitigating the reconstruction errors caused by overexposure on highly reflective surfaces-an issue common in traditional techniques. The proposed approach provides complete and reliable 3D morphological information to support robotic arm manipulation. Experimental results confirm that the method accurately captures the 3D profiles of reflective components such as CPUs and motherboards. Moreover, validation across computer enclosures of different brands and form factors demonstrates the method's robustness and practical applicability in a wide range of e-waste disassembly scenarios. (c) 2025 Optica Publishing Group. All rights, including for text and data mining (TDM), Artificial Intelligence (AI) training, and similar technologies, are reserved.
...4.FDCGAN: frequency domain constrained generative adversarial network for overexposed fringe image restoration
- 关键词:
- 3D reconstruction;Frequency domain analysis;Interferometry;Profilometry;Restoration;Three dimensional computer graphics;Adversarial networks;Domain learning;Frequency domains;Fringe images;Fringe pattern;Fringe projection profilometry;Hierarchical features;Severe saturations;Spatial and frequency domain;Structural details
- Cheng, Yang;Li, Jiaqiong;Balasubramaniam, Badrinath;Li, Beiwen
- 《Optics Express》
- 2025年
- 33卷
- 17期
- 期刊
Restoring overexposed fringe images in fringe projection profilometry (FPP) is a challenging task due to severe saturation and loss of fine structural details, which often renders traditional methods ineffective. In this work, we propose FDCGAN, a Frequency Domain Constrained Generative Adversarial Network designed to address this issue by combining spatial and frequency-domain learning. The proposed framework embodies a structurally coherent design that couples hierarchical feature encoding with multi-scale adversarial supervision, enabling both global structure preservation and local detail recovery, while normalization strategies regulate information flow and enhance resilience to photometric degradation. To further enhance restoration quality, we introduce a set of frequency-aware learning losses, including Fourier magnitude loss and high-frequency loss, which guide the model in reconstructing realistic fringe patterns. Experimental results demonstrate that FDCGAN significantly outperforms existing methods, especially under extreme overexposure, where fringe patterns and object shapes are barely visible. Even in such cases, FDCGAN can infer plausible fringe structures that significantly enhance the quality of 3D reconstruction. These findings highlight FDCGAN’s robustness and its potential for improving measurement reliability in real-world FPP systems under challenging lighting conditions. © 2025 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement.
...5.Comparative analysis of circular and linear fringe projection profilometry: from calibration to 3D reconstruction
- 关键词:
- Image reconstruction;Profilometry;3D reconstruction;Calibration method;Comparative analyzes;Fringe pattern;Fringe projection profilometry;Linear phasis;Performance;Phase shifting methods;Reconstruction techniques;System calibration
- Li, Jiaqiong;Li, Beiwen
- 《Optics Continuum》
- 2024年
- 3卷
- 3期
- 期刊
This study compares the accuracy of circular and linear fringe projection profilometry in the aspects of system calibration and 3D reconstruction. We introduce, what we believe to be, a novel calibration method and 3D reconstruction technique using circular and radial fringe patterns. Our approach is compared with the traditional linear phase-shifting method through several 2 × 2 experimental setups. Results indicate that our 3D reconstruction method surpasses the linear phase-shifting approach in performance, although calibration efficiency does not present a superior performance. Further analysis reveals that sensitivity and estimated phase error contribute to the relative underperformance in calibration. This paper offers insights into the potentials and limitations of circular fringe projection profilometry. © 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement.
...6.TPDNet: Texture-Guided Phase-to-DEPTH Networks to Repair Shadow-Induced Errors for Fringe Projection Profilometry
- 关键词:
- fringe projection profilometry; shadow-induced errors; texture-guidedphase-to-depth networks;STRUCTURED LIGHT
- Li, Jiaqiong;Li, Beiwen
- 《PHOTONICS》
- 2023年
- 10卷
- 3期
- 期刊
This paper proposes a phase-to-depth deep learning model to repair shadow-induced errors for fringe projection profilometry (FPP). The model comprises two hourglass branches that extract information from texture images and phase maps and fuses the information from the two branches by concatenation and weights. The input of the proposed model contains texture images, masks, and unwrapped phase maps, and the ground truth is the depth map from CAD models. A loss function was chosen to consider image details and structural similarity. The training data contain 1200 samples in the verified virtual FPP system. After training, we conduct experiments on the virtual and real-world scanning data, and the results support the model's effectiveness. The mean absolute error and the root mean squared error are 1.0279 mm and 1.1898 mm on the validation dataset. In addition, we analyze the influence of ambient light intensity on the model's performance. Low ambient light limits the model's performance as the model cannot extract valid information from the completely dark shadow regions in texture images. The contribution of each branch network is also investigated. Features from the texture-dominant branch are leveraged as guidance to remedy shadow-induced errors. Information from the phase-dominant branch network makes accurate predictions for the whole object. Our model provides a good reference for repairing shadow-induced errors in the FPP system.
...
