人工视网膜假体视觉下图像信息优化策略及其认知机制研究

项目来源

国家自然科学基金(NSFC)

项目主持人

柴新禹

项目受资助机构

上海交通大学

立项年度

2017

立项时间

未公开

项目编号

61773256

研究期限

未知 / 未知

项目级别

国家级

受资助金额

64.00万元

学科

信息科学-人工智能-认知与神经科学启发的人工智能

学科代码

F-F06-F0609

基金类别

面上项目

关键词

视觉计算 ; 视觉信息处理 ; 视网膜假体 ; 深度知觉 ; 视觉注意 ; visual information processing ; visual computing ; retinal prothesis ; visual attention ; depth perception

参与者

陆燕玉;李恒;芦卓凡;李谦;曹晓飞;苏小帆;纪晓勇

参与机构

上海交通大学

项目标书摘要:人工视网膜假体已成为盲人视觉功能修复的有效手段。然而,由于受到当前制造技术的制约,可植入的电极数量受限,当前假体植入者仅能获得低分辨率、低灰度级、缺乏颜色信息的人工视觉,因此植入者的视觉感受中缺失了大部分的纹理、对比度及空间深度等特征信息;同时,由于受到视网膜凹形结构和手术安全的限制,电极阵列尺寸不能过大,目前只覆盖黄斑区以内的视网膜区域,导致假体植入者仅能获得小视野的假体视觉。针对这些关键问题,本课题基于视觉科学、认知科学及信息科学,拟开展三方面的研究:复杂场景下基于场景对象分级的假体视觉呈现;扩大假体视觉感知视野的图像处理策略及其空间认知机制;假体视觉下的空间深度感知及其优化编码策略。旨在通过假体视觉下的信息优化处理及认知机制研究,改善假体植入者的视觉感受,提高其完成视觉任务的能力,为视网膜假体中的图像处理编码方案提供重要的理论基础,同时,也为视觉功能修复的认知机制奠定科学的实验依据。

Application Abstract: Artificial retinal prostheses have been an effective approach for restoring the blind’s visual function.However,due to the constraints from manufacturing technology,the implantable electrode number is limited,current prosthesis recipients only obtain low-resolution,low-brightness,and color-lost prosthetic vision perception.Therefore,a large amount of the feature information such as texture,contrast,and spatial depth information has lost in the recipients’visual experience.Meanwhile,due to the limitations in concave structure of the retina and surgical safety,the size of the electrode array in a retinal prosthesis can only cover a small area of the retina(within the macular area),leading to prosthesis recipients only sighting a small visual field.For these problems,based on visual science,cognitive science and information science,this research project will carry out three studies:prosthetic vision rendering based on scenario-object-classification under complex scenes;image processing strategies for expanding the perceived visual field of prosthetic vision and its spatial cognitive mechanism;and spatial depth perception and its encoding optimization strategies under prosthetic vision.The objective is to improve prosthesis recipients’vision perception,enhancing their ability to accomplish visual tasks,and contributing to the design of information processing modules for retinal prostheses by information optimization processing and the cognitive mechanism research under prosthetic vision.At the same time,it will also provide scientific experimental basis for the cognitive mechanism of visual function restoration.

项目受资助省

上海市

项目结题报告(全文)

失明是对人类生活质量影响最严重的一种残疾。通过电刺激作为神经调控手段来刺激残存的视网膜神经元,从而实现视觉功能的修复已经成为神经科学与工程领域新的研究热点。视觉功能修复在取得重大进展的同时,也存在着视觉感受分辨率低、视野小等瓶颈问题。本课题针对上述关键科学技术问题,创新地提出了利用有限数量的微电极实现高效的功能性假体视觉。为了实现这一目标,本课题开展了一系列富有成效的研究工作,首先,针对复杂场景的视觉任务,建立了基于场景对象分级的显著性检测与分割模型,实现了场景中不同重要度图像信息的分级呈现,使得假体植入者通过有限数量的光幻视点获得更加丰富的视觉信息,提高其完成复杂视觉任务的能力;在此研究基础上,针对当前视网膜假体临床应用中的视野小问题,开展扩大假体视觉感知视野的图像处理策略研究,通过基于图像内容重定位的研究思路,尽可能保留场景中的重要信息,并压缩不重要的背景信息,使得假体植入者在有限的视野范围内能够获得更多重要的视觉信息,从而实现扩大假体视觉感知视野的目标。同时,探索扩大假体视觉感知视野下的空间认知机制,为假体视觉下大脑可塑性的研究提供科学的实验依据;此外,进一步建立了适用于假体视觉的单目深度估计神经网络模型,并在公开数据集上对模型的性能进行了定性和定量的评估,验证了模型的有效性。其研究成果不仅能够改善假体植入者的视觉感受,同时,也为视觉功能修复的认知机制奠定重要的实验依据和理论基础。

  • 排序方式:
  • 2
  • /
  • 1.An infrared image-enhancement algorithm in simulated prosthetic vision: Enlarging working environment of future retinal prostheses

    • 关键词:
    • dual-mode camera; infrared image-processing algorithm; retinalprosthesis; simulated prosthetic vision;ARTIFICIAL VISION; OBJECT RECOGNITION; PERFORMANCE; STRATEGIES;CHARACTERS; IMPLANTS; MODEL
    • Liang, Junling;Li, Heng;Chen, Jianpin;Zhai, Zhenzhen;Wang, Jing;Di, Liqing;Chai, Xinyu
    • 《ARTIFICIAL ORGANS》
    • 2022年
    • 46卷
    • 11期
    • 期刊

    Background Most existing retinal prostheses contain a built-in visible-light camera module that captures images of the surrounding environment. Thus, in case of insufficient or lack of visible light, the camera fails to work, and the retinal prostheses enter a dormant or "OFF" state. A simple and effective solution is replacing the visible-light camera with a dual-mode camera. The present research aimed to achieve two main purposes: (1) to explore whether the dual-mode camera in prosthesis recipients works under no visible-light conditions and (2) to assess its performance. Methods To accomplish these aims, we enrolled subjects in a psychophysical experiment under simulated prosthetic vision conditions. We found that the subjects could complete some simple visual tasks, but the recognition performance under the infrared mode was significantly inferior to that under the visible-light mode. These results inspired us to develop and propose a feasible infrared image-enhancement processing algorithm. Another psychophysical experiment was performed to verify the feasibility of the algorithm. Results The obtained results showed that the average efficiency of the subjects completing visual tasks using our enhancement algorithm (0.014 +/- 0.001) was significantly higher (p < 0.001) than that of subjects using direct pixelization (0.007 +/- 0.001). Conclusions We concluded that a dual-mode camera could be a feasible solution to improving the performance of retinal prostheses as the camera adapted better to the specific existing ambient light conditions. Dual-mode cameras combined with this infrared image-enhancement algorithm could provide a promising direction for the design of future retinal prostheses.

    ...
  • 3.Objective neuromodulation basis for intrafascicular artificial somatosensation through carbon nanotube yarn electrodes

    • 《JOURNAL OF NEUROSCIENCE METHODS》
    • 2022年
    • 369卷
    • 期刊

    Background: Intrafascicular electrical stimulation has been extensively adopted to achieve sensory feedback for limb amputees. Axon-like carbon nanotube yarn (CNTy) electrodes with both promising flexibility and spatial selectivity index (SSI) can be fascinating alternatives to generate artificial somatosensation. New method: Here we systematically disclose objective neuromodulation basis for artificial somatosensation through intrafascicular CNTy electrodes. CNTy electrodes with different exposed lengths were utilized for electrically stimulating tibial nerves in twelve rats. Somatosensory evoked potentials (SEPs) were recorded synchronously using an epidural thirty-channel electrode array. Spatiotemporal characteristics of SEPs were analyzed as current pulse amplitude (PA), pulse width (PW) and pulse frequency (PF) varied. Results: The current thresholds at 1 Hz exhibit the lowest means when compared with those at 4 and 8 Hz for most CNTy electrodes (20/28). For all the electrodes, amplitudes of SEPs and activated areas of perceptive fields increase with PWs and PAs rising, and decrease remarkably with PFs from 1 to 8 Hz. Latencies of P1 and N1 of SEP peaks gradually reduced with PWs and PAs advancing. Considering high SSIs, relatively stable current thresholds, wider variation ranges of sensory magnitudes and optimal stability of perceptive fields, the L-200 mu m electrodes are preferable for neuromodulation with PFs of 1 - 8 Hz, PWs of 100 - 800 mu s and PAs of 2 - 64 mu A. Comparison with existing methods: New-type CNTy electrodes possess both promising flexibility and SSI when compared with other neural interfaces. We systematically explore objective neuromodulation basis for artificial somatosensation through CNTy electrodes for the first time. Conclusions: Significantly higher SSIs, lower current and charge thresholds exist for CNTy electrodes in comparison with other peripheral-nerve interfaces. This study can, for the first time, lay a solid neuromodulation foundation for CNTy electrodes to achieve fine sensory feedback.

    ...
  • 4.Computational Modeling of Spatially Selective Retinal Stimulation with Temporally Interfering Electric Fields

    • 关键词:
    • Aldehydes;Computation theory;Ophthalmology;Electric fields;Computational model;Electrical stimulations;High intensity regions;Multiple electrodes;Neuroprotective effects;Retinal degenerative disease;Retinal stimulation;Temporal interference
    • Su, Xiaofan;Guo, Jiahui;Zhou, Meixuan;Chen, Jianpin;Li, Liming;Chen, Yao;Sui, Xiaohong;Li, Heng;Chai, Xinyu
    • 《IEEE Transactions on Neural Systems and Rehabilitation Engineering》
    • 2021年
    • 29卷
    • 期刊

    Retinal electrical stimulation is a widely utilized method to restore visual function for patients with retinal degenerative diseases. Transcorneal electrical stimulation (TES) represents an effective way to improve the visual function due to its potential neuroprotective effect. However, TES with single electrode fails to spatially and selectively stimulate retinal neurons. Herein, a computational modeling method was proposed to explore the feasibility of spatially selective retinal stimulation via temporally interfering electric fields. An eyeball model with multiple electrodes was constructed to simulate the interferential electric fields with various electrode montages and current ratios. The results demonstrated that the temporal interference (TI) stimulation would gradually generate an increasingly localized high-intensity region on retina as the return electrodes moved towards the posterior of the eyeball and got closer. Additionally, the position of the convergent region could be modulated by regulating the current ratio of different electrode channels. The TI strategy with multisite and steerable stimulation can stimulate local retinal region with certain convergence and a relatively large stimulation range, which would be a feasible approach for the spatially selective retinal neuromodulation.
    © 2001-2011 IEEE.

    ...
  • 5.Modulation of Spike Count Correlations Between Macaque Primary Visual Cortex Neurons by Difficulty of Attentional Task

    • 关键词:
    • Rhesus monkey; Spatial attention; Shared variability; Attentional load;Primary visual cortex;SELECTIVE ATTENTION; NOISE CORRELATIONS; TEMPORAL SCALES; SINGLENEURONS; V4; CONNECTIVITY; FLUCTUATIONS; STIMULATION; VARIABILITY;COMPETITION
    • Hu, Qiyi;Hu, Wenjuan;Liu, Keyi;Bu, Xiangdong;Hu, Lisha;Li, Liming;Chai, Xinyu;Chen, Yao
    • 《NEUROSCIENCE BULLETIN》
    • 2021年
    • 38卷
    • 5期
    • 期刊

    Studies have shown that spatial attention remarkably affects the trial-to-trial response variability shared between neurons. Difficulty in the attentional task adjusts how much concentration we maintain on what is currently important and what is filtered as irrelevant sensory information. However, how task difficulty mediates the interactions between neurons with separated receptive fields (RFs) that are attended to or attended away is still not clear. We examined spike count correlations between single-unit activities recorded simultaneously in the primary visual cortex (V1) while monkeys performed a spatial attention task with two levels of difficulty. Moreover, the RFs of the two neurons recorded were non-overlapping to allow us to study fluctuations in the correlated responses between competing visual inputs when the focus of attention was allocated to the RF of one neuron. While increasing difficulty in the spatial attention task, spike count correlations were either decreased to become negative between neuronal pairs, implying competition among them, with one neuron (or none) exhibiting attentional enhancement of firing rate, or increased to become positive, suggesting inter-neuronal cooperation, with one of the pair showing attentional suppression of spiking responses. Besides, the modulation of spike count correlations by task difficulty was independent of the attended locations. These findings provide evidence that task difficulty affects the functional interactions between different neuronal pools in V1 when selective attention resolves the spatial competition.

    ...
  • 8.A Three-Dimensional Microelectrode Array to Generate Virtual Electrodes for Epiretinal Prosthesis Based on a Modeling Study

    • 关键词:
    • Microelectrodes;Vision;Prosthetics;Electric fields;Aldehydes;Distribution of electric fields;Electric field modulation;Electrical stimulations;Epiretinal prosthesis;Microelectrode array;Spatial localization;Stimulation strategies;Virtual electrode
    • Lyu, Qing;Lu, Zhuofan;Li, Heng;Qiu, Shirong;Guo, Jiahui;Sui, Xiaohong;Sun, Pengcheng;Li, Liming;Chai, Xinyu;Lovell, Nigel H.
    • 《International Journal of Neural Systems》
    • 2020年
    • 30卷
    • 3期
    • 期刊

    Despite many advances in the development of retinal prostheses, clinical reports show that current retinal prosthesis subjects can only perceive prosthetic vision with poor visual acuity. A possible approach for improving visual acuity is to produce virtual electrodes (VEs) through electric field modulation. Generating controllable and localized VEs is a crucial factor in effectively improving the perceptive resolution of the retinal prostheses. In this paper, we aimed to design a microelectrode array (MEA) that can produce converged and controllable VEs by current steering stimulation strategies. Through computational modeling, we designed a three-dimensional concentric ring-disc MEA and evaluated its performance with different stimulation strategies. Our simulation results showed that electrode-retina distance (ERD) and inter-electrode distance (IED) can dramatically affect the distribution of electric field. Also the converged VEs could be produced when the parameters of the three-dimensional MEA were appropriately set. VE sites can be controlled by manipulating the proportion of current on each adjacent electrode in a current steering group (CSG). In addition, spatial localization of electrical stimulation can be greatly improved under quasi-monopolar (QMP) stimulation. This study may provide support for future application of VEs in epiretinal prosthesis for potentially increasing the visual acuity of prosthetic vision.
    © 2020 World Scientific Publishing Company.

    ...
  • 9.Retinal neurovascular responses to transcorneal electrical stimulation measured with optical coherence tomography

    • 《EXPERIMENTAL BIOLOGY AND MEDICINE》
    • 2020年
    • 245卷
    • 4期
    • 期刊

    Noninvasive transcorneal electrical stimulation (TES) has emerged as a potential strategy to facilitate visual restoration and promote retinal cell survival for certain retinal and optic nerve diseases owing to its neuroprotective effects. However, the neurovascular responses of retinal neurons evoked by TES have not been completely determined. To investigate this issue, we utilized a custom-designed spectral-domain optical coherence tomography (SD-OCT) to record the retinal neural and vascular responses under TES in vivo simultaneously. Significant increases of both positive and negative intrinsic optical signal (IOS) changes were recorded in all three segmented retinal layers, which mainly related to neural activities. However, the changes of TES-induced retinal vascular responses, including blood velocity, cross-sectional area of vessel, and blood flow, were not significant. It suggests that TES mainly elicited neural responses in retina, while no significant vascular responses were evoked. Our results provide experimental evidence to the mechanism of retinal neurovascular coupling under TES. Additionally, the present study also suggests that SD-OCT could be utilized as a promoting method to explore neurovascular responses under retinal stimulation in clinical treatment and technology. Impact statement Noninvasive transcorneal electrical stimulation (TES) has emerged as an effective treatment for certain retinal and optic nerve diseases owing to its neuroprotective effects. However, the retinal neurovascular responses evoked by TES have not been completely determined. To investigate this issue, we utilized a custom-designed spectral-domain optical coherence tomography (SD-OCT) to record the retinal neural and vascular responses evoked by TES in vivo simultaneously. The present study suggested that TES mainly elicited neural responses in retina, while no significant vascular responses were evoked. Our results provide experimental evidence to the mechanism of retinal neurovascular coupling evoked by TES. Additionally, the present study also suggests that SD-OCT could be utilized as a promoting method to explore neurovascular responses under retinal electrical stimulation.

    ...
  • 10.Eddy-current-induced distortion correction using maximum reconciled mutual information in diffusion MR imaging.

    • 关键词:
    • Diffusion MR; Diffusion-weighted imaging; Mutual information; Reconcile entropy; Registration
    • Liang, Junling;Zhao, Shujun;Di, Liqing;Wang, Jingjuan;Sun, Pengcheng;Chai, Xinyu;Li, Heng
    • 《International journal of computer assisted radiology and surgery》
    • 2019年
    • 14卷
    • 3期
    • 期刊

    PURPOSE: In diffusion tensor imaging, a large number of diffusion-weighted (DW) images with different diffusion gradient directions are attained during scanning. However, subjects' involuntary head movements and eddy current effect related to large diffusion-sensitizing gradients will cause distortions of DW images. Therefore, for tracking accurately white matter structures and tractography, the distortions have to be realigned before model fitting. Currently, traditional methods use maximum mutual information (MMI) or normalized mutual information (NMI) as similarity measure for DW images registration. These information measures are defined by Shannon entropy. The image entropy is able to embody the global information complexity but ignore the local information complexity caused by heterogeneous intensity contrasts in DW images, making registration algorithm early converge.; METHOD: To overcome the above problem, we present maximum reconciled mutual information (MRMI) combining both global information and local information as the similarity measure of the registration algorithm framework.; RESULT: (i) In comparison with traditional methods, under our proposed MRMI method, the border of DW image is more anastomotic with the b0 image, and the fitted fractional anisotropy (FA) map after registration is closer to the true brain boundary. (ii) By quantitative analysis of registration results, our method has a significant advantage over others in terms of NMI between b0 image and the aligned DW images.; CONCLUSION: The results suggest that there is a high-level matching in space between the b0 image and the DW images aligned by the MRMI method, raising the registration robustness and accuracy compared to the traditional DW registration methods. It may provide a better option for the existing diffusion image registration tools (e.g., FMRIB Software Library) and commonly multimodal medical image registration.

    ...
  • 排序方式:
  • 2
  • /