面向眼控系统交互式界面元素的视觉表征与评价机制研究

项目来源

国家自然科学基金(NSFC)

项目主持人

牛亚峰

项目受资助机构

东南大学

立项年度

2018

立项时间

未公开

项目编号

71801037

项目级别

国家级

研究期限

未知 / 未知

受资助金额

19.00万元

学科

管理科学-管理科学与工程-工业工程与质量管理

学科代码

G-G01-G0108

基金类别

青年科学基金项目

关键词

视觉表征 ; 工效学评价 ; 交互式界面元素 ; 眼控系统 ; 视觉表征 ; 工效学评价 ; 交互式界面元素 ; 眼控系统

参与者

周蕾;张晶;汤文哲;王欣悦;肖玮烨;李贤;罗子敬

参与机构

上海理工大学;南京林业大学;东南大学

项目标书摘要:随着计算机技术及人工智能的迅猛发展,符合人类自然认知习惯的眼控技术成为人机交互发展的主流技术之一。交互式界面元素成为眼控系统的第一交互入口,其视觉表征直接影响眼机交互效率。本项目以贴近人类认知本源属性的视角开展眼控系统交互式界面元素的视觉表征和交互设计,研究眼动生理特征形式和交互式界面元素的映射机理,寻求交互式界面元素的眼睛动作流设计和最佳动作组合,探索交互式界面元素的工效学评价指标,分析“Midas接触”和“低空间精确度”的设计规范及其规避策略,以期建立一套科学的面向眼控系统交互式界面元素视觉表征优化设计方法与工效学评价机制,为人工智能、先进交互环境下开展“眼机交互—界面设计”研究提供理论支撑和技术手段。研究成果可应用于军事眼控装备系统中去,具有广泛的应用前景,也可促进设计学、工效学与眼机交互领域的交叉融合,具有显著的理论意义和学术价值。

Application Abstract: With the rapid development of computer technology and artificial intelligence,eye-control technology,which conforms to the natural cognition habit of human,has become one of the mainstream technologies of human-computer interaction development.Interactive interface element is the first interactive entry of eye-control system,and its visual representation directly affects the interaction efficiency.This project carries out visual representation and interaction design of the eye control system interactive interface elements from the perspective of human cognitive origin attribute.Mapping mechanism between eye movement physiological characteristics form and interactive interface elements,and eye movement flow design and best action combinations for interactive interface elements are studied.The ergonomic evaluation index of interactive interface elements is explored.Design specification and its countermeasure of"Midas touch"and"low precision space"is analyzed.In order to establish a set of visual representation optimization design method and ergonomic evaluation mechanism for eye control system interactive interface element,providing theoretical support and technical means for the research of"eye-machine interaction-interface design"in artificial intelligence and advanced interactive environment.Research results can be applied to military eye control equipment system,have extensive application prospect,can promote the intersection of design,ergonomics and eye-machine interaction,have significant theoretical meaning and academic value.

项目受资助省

江苏省

项目结题报告(全文)

本项目主要完成了下述工作,并取得了相应研究成果:分析了交互式界面元素的图形符号、色彩、反馈等信息编码规律,探究了图形符号、色彩搭配、交互反馈等成分的最佳组织形式和表征规则。建立了交互式界面元素的布局形态设计体系,完善了眼控系统界面设计功能分区及符合视觉运动规律的界面布局。完成了对界面元素有效交互面积、间距和交互距离的工效学研究,提出了眼控系统交互式界面元素的设计标准。构建了眨眼、凝视、扫视、平滑跟踪、闭眼等眼睛动作指令与单个交互式界面元素的自然逻辑关系和映射,完成了对眼控最佳动作组合形式和凝视触发时间的实验研究。综合运用设计科学、认知心理学、行为学等学科理论和技术手段,探究了多个交互式界面元素条件下眼睛动作流最佳纵向深度与横向容量。从设计学和工效学角度提出了解决“Midas接触”和“低空间精确度”的策略和建议,构建了面向眼控系统交互式界面元素的视觉表征优化设计策略。基于用户交互反馈过程中的用户行为和生理表现,提出了面向交互式界面元素用户认知的关键性脑电成分和脑电指标,确定了交互反馈过程中的行为指标(眼动数据、行为数据)和脑生理指标(ERP成分)的阈值范围,及其两者之间的关联性,确定了交互式界面元素的工效学评价指标。项目实施以来,项目组成员以一作/通讯作者身份共发表17篇论文,其中SCI/SSCI检索和录用7篇,EI检索和录用8篇,项目负责人以第一作者出版学术专著1本,获得授权发明专利2项,获得省部级科研奖励3项。11篇硕士学位论文受到本项目资助。此外,项目组与多家研究所和企业积极开展科研合作,并把项目研究成果推广到自闭症儿童注意力康复软件的开发中,显示了较高的应用价值。

  • 排序方式:
  • 5
  • /
  • 1.Smooth Pursuit Study on an Eye-Control System for Continuous Variable Adjustment Tasks

    • 关键词:
    • Control systems;Continuous variables;Eye control;Human eye;Important value;Optimal combination;Parameter set;Property;S-algorithms;Smooth pursuit;Three-level
    • Niu, Yafeng;Li, Xin;Yang, Wenjun;Xue, Chengqi;Peng, Ningyue;Jin, Tao
    • 《International Journal of Human-Computer Interaction》
    • 2023年
    • 39卷
    • 1期
    • 期刊

    Smooth pursuit is a physiological phenomenon of human eyes, which can be used as eye-control interaction to avoid the "Midas" problem. This article describes the smooth pursuit properties of continuous variable tasks under Euclid’s algorithm. Two experiments were conducted to investigate the effects of slider speed, initial position of "plus," and variable value position on smooth pursuit interactions. The initial position and slider speed are combined in 12 combination forms, and the variable value positions are divided into 7 cases. Then, the optimal combination forms and variable value position case are derived. Experiment 1 divided the initial position into four levels of up, down, left and right and divided the slider speed into three levels of 1 deg/s, 2 deg/s, and 3 deg/s, concluding that the optimal combination of initial position and slider speed is that the speed is 2 deg/s and initial position is up. Experiment 2 was conducted on the basis of the conclusion obtained in experiment 1. The position of the variable value was divided into seven cases according to whether the left, center, and right of the smooth pursuit track the variable value appeared. The result showed that the position of the variable value in the center had the greatest influence on the pursuit efficiency. This research has an important value for the parameter set of smooth pursuit in the case of the Euclidean algorithm, it has great guiding significance for building smooth pursuit interaction in the context of the eye-control system.
    © 2021 Taylor & Francis Group, LLC.

    ...
  • 2.基于稳态视觉诱发电位的脑机接口元素尺寸和间距工效学研究

    • 关键词:
    • 脑机接口;稳态视觉诱发电位;工效学;尺寸;间距
    • 牛亚峰;王佳浩;伍金春;薛澄岐;杨文骏
    • 《电子与信息学报》
    • 2022年
    • 2期
    • 期刊

    针对基于稳态视觉诱发电位(SSVEP)的脑机接口系统,该文开展了屏显刺激界面元素尺寸和间距对识别效率和用户体验影响的工效学实验研究。该工效学实验使用红色正方形作为频闪刺激元素,刺激元素位于上、下、左、右等4个位置,自变量包含尺

    ...
  • 4.Design recommendations for voluntary blink interactions based on pressure sensors

    • 关键词:
    • Efficiency;Human computer interaction;Product design;Signal processing;Blink detections;Design recommendations;Eye-controled interaction;Interaction design;Low-power consumption;Lower-power consumption;Materials processing;Sensor materials;Sensor signals;Voluntary blink
    • Fan, Lin-Han;Huang, Wei-Chi;Shao, Xue-Qi;Niu, Ya-Feng
    • 《Advanced Engineering Informatics》
    • 2024年
    • 61卷
    • 期刊

    Voluntary blink control is vital in eye-controlled human–computer interaction (ECHCI). Pressure sensors, known for their low power consumption and strong resistance to interference, are suitable for blink detection. Existing research has predominantly focused on sensor materials and signal processing, and studies on the characteristics and optimized design of voluntary blink actions in ECHCI remain relatively limited. Therefore, this study presents a blink detection system based on pressure sensors and investigates the efficiency of voluntary blink actions. We examined the effects of two independent variables, blink count (single, double, and triple) and blink side (bilateral and unilateral), on recognition accuracy and three time-related variables. Subsequently, the subjective workload and evaluation experiments were conducted for each blink action. The results indicate that the blink count has a more pronounced impact on interaction efficiency and comfort, with continuous blink actions exceeding three blinks not being recommended. Bilateral blinks were perceived as more natural and comfortable than unilateral blinks. Moreover, we discovered that the participants exhibited greater attentiveness and focus when performing double bilateral blink actions, making it suitable for important commands. Based on the experimental results, the top three recommended voluntary blink actions are single bilateral blink (SB), double bilateral blink (DB), and single unilateral blink (SU). Finally, a test involving controlling the movement of a toy car using the recommended blink actions validates the usability of the actions and detection system. These findings provide valuable guidance for the design and optimization of voluntary-blink interaction systems, further enhancing this interaction method's feasibility. © 2024 Elsevier Ltd

    ...
  • 5.Magilock: a reliable control triggering method in multi-channel eye-control systems

    • 关键词:
    • eye control; multi-channel interaction; Midas touch; lock time; unlocktime;GAZE
    • Niu, Ya-Feng;He, Jia-Xin;Jin, Liu
    • 《FRONTIERS IN HUMAN NEUROSCIENCE》
    • 2024年
    • 18卷
    • 期刊

    Eye-tracking technology brings a different human-computer interaction experience to users because of its intuitive, natural, and hands-free operation characteristics. Avoiding the Midas touch problem and improving the accuracy of interaction are among the main goals of the research and development of eye-control systems. This study reviews the methods and limitations of research on avoiding the Midas touch problem. For typical control clicking operations with low fault tolerance, such as mode switching and state selection in an eye-control system, this study proposes Magilock, a more reliable control triggering method with a high success rate in multi-channel eye-control systems. Magilock adds a control pre-locked mechanism between the two interactive steps of eye-control channel positioning control and other interactive channel triggering controls in the multi-channel eye-control system. This effectively avoids incorrect control triggering caused by multi-channel coordination disorder and gaze-point drift. This study also conducted ergonomic experiments to explore the lock and unlock times of the control pre-locked mechanism in Magilock. Taking into account the experimental data and subjective evaluation of the participants, we recommend setting the lock time and the unlock time of Magilock to 200 ms.

    ...
  • 6.Design recommendations of target size and tracking speed under circular and square trajectories for smooth pursuit with Euclidean algorithm in eye-control system

    • 关键词:
    • Control systems;Ergonomics;Target tracking;Design recommendations;Euclidean algorithms;Eye control;Eye-control system;Size;Smooth pursuit;Target size;Target speed;Targets tracking;Trajectory shapes
    • Yi-yan, Wang;Wen, Cheng;Jing-ze, Tian;Wei-chi, Huang;Ya-feng, Niu;Wen-jun, Yang;Cheng-qi, Xue
    • 《Displays》
    • 2024年
    • 81卷
    • 期刊

    In this study, we investigated how the size, speed, and trajectory shape of smooth-pursuit targets for Euclid's algorithm affect the ergonomics and usability of the gaze interface, and conducted a user study with 20 participants, examining the priority of five target sizes in circular and square trajectories, and the recommended speed for each size in the five target speeds. The usability of the SUS and ISO 9241 (acceptability and comfort) of the eye control system had been evaluated. Based on the experiment results, the design recommendations of the priority of the target size and speed of the circular and square trajectories, suggested trajectory shapes. The research is an important reference for the design and development of smooth-pursuit targets based on the Euclidean algorithm. © 2023 Elsevier B.V.

    ...
  • 7.Research on a spatial–temporal characterisation of blink-triggered eye control interactions

    • 关键词:
    • Ergonomics;Locks (fasteners);Eye blink;Eye control;Human computer interfaces;Interface system;Line of Sight;Lines-of-sight;Locking range;Mida contact;Spatial temporals;User groups
    • Wang, Yi-Yan;Tian, Jing-Ze;Xiao, Lang;He, Jia-Xin;Niu, Ya-Feng
    • 《Advanced Engineering Informatics》
    • 2024年
    • 59卷
    • 期刊

    Eye-controlled human–computer interface (ECHCI) system is gradually becoming the focus of audience attention, owing to its humanistic, natural, and intuitive characteristics, which provide services for a wider range of user groups. We conducted a study on the ECHCI with "gaze locking + blink trigger" as the interaction method in the temporal dimension and the spatial dimension respectively and explored the optimisation of blink trigger actions. First, the optimal lock time of the interactive object (IO) was 650 ms, and a subjective evaluation experiment confirmed that the user's subjective feeling was consistent with the results of the ergonomic experiment. Second, the direction and distance of the visual drift of the blink action were investigated by increasing the locking range of multiple IOs. The results showed that the main direction of the line-of-sight drift for the blinking action was the upper-right corner. Furthermore, it was experimentally demonstrated that the line-of-sight drifted by a certain distance during blinking and that the system had the highest success rate when the locking range corresponded to a viewing angle of 21.074°. This study provides theoretical support for the application of multi-movement combinations in blink ECHCI, which enhances the user experience of blink triggering. © 2023 Elsevier Ltd

    ...
  • 8.Study on the brightness and graphical display object directions of the Single-Gaze-Gesture user interface

    • 关键词:
    • Eye movements;Graphical user interfaces;Brightness values;Direction;Gesture interfaces;Graphic display object;Graphical displays;Graphical user interface;Graphics display;Interface designs;Performance;Single gaze gesture
    • Zuo, Hong-rui;Niu, Ya-feng;Tian, Jing-ze;Yang, Wen-jun;Xue, Cheng-qi
    • 《Displays》
    • 2023年
    • 80卷
    • 期刊

    Single gaze gestures (SGGs), as the basic unit of gaze gestures, interact with graphical display objects (GDOs) on a graphical user interface (GUI) through user saccades to control the system. The brightness design of the interface and saccade direction can significantly affect interaction performance and user experience. Therefore, this study adopts the method of ergonomic experiments combined with subjective evaluation, proposing the design suggestions for the background brightness (BB), brightness contrast (BC), GDO (completion field) directions, and target brightness (TB) of the gaze gesture interface: (1) Background brightness: To ensure the efficiency and experience in actual use, the background of the interface should choose a brightness value slightly lower than 50%; (2) Brightness contrast: At 25% BB, it is advisable to choose a brightness value between 21.5% and 43.5%, and the best choice is 33.25%. Under the condition of BB above 50%, the brightness contrast value of 50% can meet the requirements of the operation task; (3) Target brightness: In the gaze gesture interface design, light color should be used to mark the target; (4) GDO Directions: The GDO should be placed on the upper side of the initiation field (IF), and it is best to be directly above the IF. The contribution of this research is the proposal of an interface design strategy that is beneficial for improving the performance and experience of gaze gesture interaction and can provide future directions and recommendations for interface design. © 2023 Elsevier B.V.

    ...
  • 9.Image Entropy-Based Interface Evaluation Method for Nuclear Power Plants.

    • 关键词:
    • evaluation method; image entropy; interface; nuclear power plants
    • Tang, Wenzhe;Chen, Shanguang;Lin, Yun;Xue, Chengqi
    • 《Entropy 》
    • 2023年
    • 25卷
    • 12期
    • 期刊

    The digital interface is crucial for nuclear plant operators, influencing their decision-making significantly. However, evaluations of these interfaces often overlook users' decision-making performance; lack established standards, typically occurring after the design phase; and are unsuitable for large-scale assessments. Recognizing the vital role of interface information, this paper built on our previous research and proposed a method tailored for nuclear power plant interfaces, utilizing image entropy to evaluate the impact of information on decision-making. A comparative analysis with an experimental evaluation method empirically validated the effectiveness of the proposed method. This research offers a unique decision-making-centric method to interface evaluation, providing a standardized, adaptable framework for various design phases and enabling extensive and rapid evaluations.

    ...
  • 10.Effects of speed, motion type, and stimulus size on dynamic visual search: A study of radar human-machine interface

    • 关键词:
    • Dynamic visual search; Visual perception; Human-machine interface;Flicker motion;DISPLAY MOVEMENT; ATTENTION; SYSTEM; PERFORMANCE; DESIGN; ICON; BLUR;UNCERTAINTY; PERCEPTION; CONTRAST
    • Tong, Mu;Chen, Shanguang;Niu, Yafeng;Xue, Chengqi
    • 《DISPLAYS》
    • 2023年
    • 77卷
    • 期刊

    Human visual perception behavior in dynamic environments could be affected by the motion properties of stimuli. The purpose of this study was to investigate the effects of motion type, size, and speed of stimulus on visual search performance and visual comfort in both single-and double-target tasks. Twenty participants attended an experiment in which they finished two tasks at three speeds (low, medium, and high), in two motion types (flicker, uniform), with varied levels of stimulus size (small, medium, and large) on a display. The search time, accuracy, critical flicker frequency, visual comfort, and cognitive load were recorded to provide a comprehensive understanding of participants' visual preferences during the experiment. The results indicated the increasing speed would weaken participants performance on visual search tasks; Medium-sized stimuli were recommended considering search efficiency and cost-effectiveness; More impor-tantly, the present study also found that flicker motion led to better task performance in the low-speed task, due to increased alertness and lead physiological arousal effects, which may imply the past studies have overstated the negative effects of flickering in dynamic environments.

    ...
  • 排序方式:
  • 5
  • /