Article
Computer Science, Theory & Methods
Alexander Schaefer, Gerd Reis, Didier Stricker
Summary: Remote collaboration systems play a crucial role in modern society, especially during times of physical distancing. This article provides an overview and taxonomy of such systems, highlighting their main contributions and offering valuable insights for researchers.
ACM COMPUTING SURVEYS
(2023)
Article
Computer Science, Software Engineering
Catarina G. G. Fidalgo, Yukang Yan, Hyunsung Cho, Mauricio Sousa, David Lindlbauer, Joaquim Jorge
Summary: The recent pandemic, war, and oil crises have led to a reevaluation of travel for education, training, and meetings. Remote assistance and training have become increasingly important, and Mixed Reality (MR) offers opportunities to improve these processes. A survey of remote assistance and training in MR environments provides insights into current approaches, benefits, and challenges. The findings highlight gaps and opportunities for further research and development in this field.
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
(2023)
Article
Computer Science, Software Engineering
Boram Yoon, Jae-eun Shin, Hyung-il Kim, Seo Young Oh, Dooyoung Kim, Woontack Woo
Summary: This study compared the effects of avatars with different transparency levels on the social presence of users in task-centric MR remote collaboration in AR and VR. Results showed that avatars with a strong visual presence are not necessary when task completion is prioritized over social interaction. However, AR users preferred more vivid avatars than VR users.
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
(2023)
Article
Automation & Control Systems
Hanzhong Xu, Dianliang Wu, Yu Zheng, Qihang Yu, Yue Zhao
Summary: The study aims to address the challenges faced in production commissioning by leveraging the capabilities of augmented reality (AR) technology and digital twin technology in the context of Industry 4.0. A novel virtual-real fusion commissioning method utilizing AR is proposed, enabling real-time commissioning and fusion display of the production process. The method was applied to the operation and commissioning of a specific production unit, resulting in significant reduction in commissioning cycle and space cost savings.
INTERNATIONAL JOURNAL OF ADVANCED MANUFACTURING TECHNOLOGY
(2023)
Article
Automation & Control Systems
Zenglei Wang, Shusheng Zhang, Xiaoliang Bai
Summary: This study proposed a novel platform for remote collaboration and compared the effectiveness of sharing head pointer and AR annotations in industrial assembly tasks. The results showed that sharing head pointer can improve performance and enhance user collaborative experiences, while gaze cues (GC) in low-cost head tracking can serve as a good referential pointing.
INTERNATIONAL JOURNAL OF ADVANCED MANUFACTURING TECHNOLOGY
(2021)
Article
Computer Science, Software Engineering
Huayuan Tian, Gun A. A. Lee, Huidong Bai, Mark Billinghurst
Summary: In this paper, we investigate how virtual replicas can improve Mixed Reality (MR) remote collaboration by reconstructing the 3D task space. Our research focuses on enhancing the understanding of remote experts' intentions and instructions by using virtual replicas as spatial communication cues. By segmenting and creating virtual replicas of physical task objects, the remote user can effectively explain and guide their partner in completing the task. Our user study results demonstrate that virtual replica manipulation is more efficient than 3D annotation drawing in MR remote collaboration scenarios. We discuss the findings, limitations, and future research directions in this paper.
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
(2023)
Article
Computer Science, Software Engineering
Huayuan Tian, Gun A. Lee, Huidong Bai, Mark Billinghurst
Summary: This paper explores the use of virtual replicas to enhance Mixed Reality (MR) remote collaboration by reconstructing the task space in 3D. It investigates how virtual replicas can serve as a spatial communication cue to improve MR remote collaboration. The study found that using virtual replica manipulation was more efficient than using 3D annotation drawing in an MR remote collaboration scenario.
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
(2023)
Article
Computer Science, Software Engineering
Catarina G. Fidalgo, Yukang Yan, Hyunsung Cho, Mauricio Sousa, David Lindlbauer, Joaquim Jorge
Summary: The recent pandemic, war, and oil crises have led to a reevaluation of the need to travel for education, training, and meetings. Remote assistance and training have become increasingly important, and Mixed Reality (MR) offers opportunities for improvement. This survey examines the current approaches, benefits, and challenges of remote assistance and training in MR environments through a systematic literature review. The findings highlight gaps and opportunities in the research area, such as exploring advanced interaction techniques and collaboration scenarios beyond one-expert-to-one-trainee.
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
(2023)
Article
Multidisciplinary Sciences
Oskari Pakari, Ricardo Lopez, Ivan Druckman, Emilee Meng, Erik Zhou, Ziang Wang, Shaun D. Clarke, Sara A. Pozzi
Summary: This article demonstrates the real-time visualization of gamma ray and neutron radiation detector data in mixed reality (MR) using Microsoft HoloLens 2 smart glasses, reducing the interpretation burden. By combining a state-of-the-art dual particle imaging system with HoloLens 2, the data is displayed directly to the user through the head-mounted MR smart glasses, providing an overlay of directional information to the user's 3D visual experience.
SCIENTIFIC REPORTS
(2023)
Review
Chemistry, Multidisciplinary
Chongyu Zhu, Yao Lu, Lixin Jiang, Yanlei Yu
Summary: Liquid crystals (LCs) are soft and intelligent materials that can adjust their properties in response to external stimuli, making them ideal for constructing soft actuators and robots. By combining LC actuators and robots with virtual reality and augmented reality technologies, a new world of mixed reality (MR) with multi-sensory interaction can be created. The development of LC actuators and robots is expected to accelerate the practical application of MR technology.
ADVANCED FUNCTIONAL MATERIALS
(2021)
Article
Computer Science, Software Engineering
Hyung-il Kim, Boram Yoon, Seo Young Oh, Woontack Woo
Summary: In this paper, a prototype system for sharing a user's hand force in MR remote collaboration on physical tasks is presented. The system uses a wearable sEMG sensor to estimate hand force and visualizes it for the expert. A user study demonstrates that sensing and sharing hand force improves the expert's awareness, perception, and social presence in the collaboration.
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
(2023)
Article
Computer Science, Information Systems
Yong-Guk Go, Ho-San Kang, Jong-Won Lee, Mun-Su Yu, Soo-Mi Choi
Summary: The development of services and applications involving drones is driving the growth and popularization of the unmanned-aerial-vehicle industry. Flying first-person-view (FPV) drones requires experience, and a remote training system can help beginners practice under expert guidance.
Review
Computer Science, Interdisciplinary Applications
Shaveta Dargan, Shally Bansal, Munish Kumar, Ajay Mittal, Krishan Kumar
Summary: Augmented Reality (AR) is a technology that modifies the perception of real-world images by overlaying digital data on them. It has diverse applications in various fields and is distinct from virtual reality.
ARCHIVES OF COMPUTATIONAL METHODS IN ENGINEERING
(2023)
Article
Chemistry, Multidisciplinary
Riccardo Monterubbianesi, Vincenzo Tosco, Flavia Vitiello, Giulia Orilisi, Franco Fraccastoro, Angelo Putignano, Giovanna Orsini
Summary: The recent advancements in digital technologies have revolutionized the field of dentistry. Augmented Reality, Virtual Reality, Mixed Reality, Artificial Intelligence, and Robotics have found various applications in education, clinical practice, and diagnostics. These technologies enhance learning, improve treatment planning, facilitate communication, and provide more accurate dental diagnoses and prognoses.
APPLIED SCIENCES-BASEL
(2022)
Article
Engineering, Industrial
Sara Aquino, Mario Rapaccini, Federico Adrodegari, Giuditta Pezzotta
Summary: This paper presents a model to identify the factors influencing the adoption of augmented reality (AR) for industrial services. The study combines literature analysis with an empirical study to explore how five large industrial companies are introducing AR for supporting technical assistance and industrial services. The authors identify four categories and 18 factors that manufacturing companies should consider when introducing AR technology for industrial services.
JOURNAL OF MANUFACTURING TECHNOLOGY MANAGEMENT
(2023)
Article
Chemistry, Analytical
Tamas D. Nagy, Tamas Haidegger
Summary: This article aims to fill the gap in the research domain of surgical subtask automation by proposing standard methodologies for performance evaluation. It discusses the potential of partial autonomy in RAMIS, evaluates current metrics for performance evaluation, introduces benchmarks in the field, and addresses Human-Machine Interface (HMI) quality, robustness, and legal and ethical issues.
Article
Engineering, Electrical & Electronic
Gabor Fichtinger, Jocelyne Troccaz, Tamas Haidegger
Summary: This article provides a comprehensive narrative of the state of the art of interventional robotic systems, emphasizing their limited impact in the medical field compared to other surgical areas. It aims to support future system developers and facilitate the translation of scientific research into applied clinical technology development.
PROCEEDINGS OF THE IEEE
(2022)
Article
Engineering, Electrical & Electronic
Tamas Haidegger, Stefanie Speidel, Danail Stoyanov, Richard M. Satava
Summary: Telesurgical robotics has achieved global clinical adoption as the first domain within medicosurgical robotics. However, its market penetration is still relatively low. The adoption of telesurgical robotics has not reached its full potential due to technical complexity and financial burden. Emerging telesurgical technologies, incorporating artificial intelligence and machine learning solutions, offer significant advantages in clinical practice.
PROCEEDINGS OF THE IEEE
(2022)
Article
Chemistry, Analytical
Josh Ehrlich, Amoon Jamzad, Mark Asselin, Jessica Robin Rodgers, Martin Kaufmann, Tamas Haidegger, John Rudan, Parvin Mousavi, Gabor Fichtinger, Tamas Ungi
Summary: This study aims to automatically detect energy events and determine electrosurgical cautery settings based on sensor data to improve surgical safety. The results demonstrate the feasibility of accurately identifying energy events in different tissue types and during surgical procedures, providing an important safety feature for surgical operations.
Article
Medicine, General & Internal
Iren A. Kopcsone Nemeth, Csaba Nador, Laszlo Szilagyi, Akos Lehotsky, Tamas Haidegger
Summary: The ability of healthcare workers to learn proper hand hygiene is an important factor in reducing hospital infections. This study found that some staff members were able to quickly learn the technique, while others struggled even with more disinfectant. Identifying these individuals helps improve training efforts.
JOURNAL OF CLINICAL MEDICINE
(2022)
Article
Medicine, General & Internal
Renata Nagyne Elek, Tamas Haidegger
Summary: This study introduces an experiment on minimally invasive surgery training to autonomously assess non-technical skills and analyze workload using sensory data and a self-rating questionnaire. The results show that autonomous time series classification can effectively classify non-technical skills and workload components.
JOURNAL OF CLINICAL MEDICINE
(2022)
Article
Computer Science, Information Systems
Anna Sudar, Adam B. Csapo
Summary: 3D virtual reality spaces provide a new and stimulating way of working with digital content, leading to performance improvements in various aspects compared to 2D interfaces. This paper reviews relevant literature and presents an exploratory study comparing 3D desktop virtual environment and 2D web-based environment through eye tracking data collection. The study identifies descriptive markers for predicting users' effectiveness in virtual reality workspaces and supports previous research claiming improved effectiveness in 3D spaces. Further work is needed to determine the constraints of such benefits.
Review
Robotics
Kristof Moga, Andrea Ferencz, Tamas Haidegger
Summary: This article provides a scoping review on the current status of Image-Guided Navigation with various forms of digital technologies, including Extended Reality, Augmented Reality Head-Mounted Displays (AR-HMDs) and Robot-Assisted Surgery (RAS) for Pedicle Screw Placement in orthopedics and spine surgery. The results showed that the accuracy of pedicle screw placement using AR-HMDs was higher compared to the traditional free-hand insertion technique, and similar outcomes were observed with RAS screwing.
Editorial Material
Automation & Control Systems
Alex Mason, Tamas Haidegger, Ole Alvseike
IEEE ROBOTICS & AUTOMATION MAGAZINE
(2023)
Article
Public, Environmental & Occupational Health
Constantinos Voniatis, Szava Bansaghi, Daniel Sandor Veres, Peter Szeremy, Angela Jedlovszky-Hajdu, Attila Szijarto, Tamas Haidegger
Summary: This study compared gel and liquid hand sanitizers in terms of application time, spillage, and coverage. The results showed that a volume of 1.5 ml was insufficient, while 3 ml resulted in almost complete coverage. Personalized, hand size-based hand sanitizer volumes may be the solution to optimize coverage and avoid spillage and long drying times.
ANTIMICROBIAL RESISTANCE AND INFECTION CONTROL
(2023)
Review
Medicine, General & Internal
Kristof Moga, Aron Holgyesi, Zsombor Zrubka, Marta Pentek, Tamas Haidegger
Summary: This research paper provides a systematic literature review on the current status of augmented-reality head-mounted devices (AR-HMDs) in guiding and navigating spine surgeries and pedicle screw placement. The analysis supports the use of AR-HMDs as a cost-effective alternative to robot-assisted surgical (RAS) systems, with similar clinical outcomes.
JOURNAL OF CLINICAL MEDICINE
(2023)
Article
Computer Science, Information Systems
Ildiko Horvath, Adam B. Csapo
Summary: In this paper, a new VR concept called 'context control' is proposed, which refers to dynamically reconfigurable VR workspaces based on the task and user's learning and working style. An implementation framework is presented, utilizing Kolb's learning styles taxonomy and providing layout schemas for different learning style categories. These schemas are validated through an experiment involving 52 university students, demonstrating that participants preferred content layouts matching their cognitive profiles. Additionally, a hierarchical schema template language is proposed for further customization.
Article
Computer Science, Information Systems
Anna Sudar, Adam B. Csapo
Summary: Studies have shown that using three-dimensional virtual reality environments can enhance multiple performance dimensions compared to using two-dimensional interfaces. However, how digital content is integrated into these environments is often based on aesthetic considerations or specific workflows. This paper presents the results of an experiment that examines user preferences for arranging digital content in three-dimensional environments, providing insights for future design methodologies and automated content arrangement solutions.
Article
Automation & Control Systems
Michael A. Houghtaling, Sandro Rama Fiorini, Nicola Fabiano, Paulo J. S. Goncalves, Ozlem Ulgen, Tamas Haidegger, Joel Luis Carbonera, Joanna Isabelle Olszewska, Brian Page, Zvikomborero Murahwi, Edson Prestes
Summary: Domain-specific ontologies play a crucial role in system design by providing a framework for meeting user-level, safety, or ethical requirements. The IEEE 7007-2021 Ontological Standard for ethically driven robotics and automation systems is the first industry standard to introduce a structure of ontologies concerning robot ethics and related fields. It offers a systems science approach to support the ethically aligned design of complex cyber-physical systems (CPSs) and robots. This article provides an overview of the main ontological commitments of the standard, their rationale, and includes use cases. It also outlines future directions for ethically aligned robotics and AI-based systems based on IEEE 7007-2021, with a focus on the rapidly growing fields of service and medical robotics.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Engineering, Multidisciplinary
Kristof Takacs, Alex Mason, Luis Eduardo Cordova-Lopez, Marta Alexy, Peter Galambos, Tamas Haidegger
Summary: Ensuring the safety of equipment, environment, and operators during robot operations is crucial. Current standards for robotic systems in the agri-food domain are lacking, with existing standards mainly focused on machinery hygiene in the meat industry. This paper aims to provide an overview of relevant regulations and standards for the development of safe robotic systems in slaughterhouses and assist in the certification process for meat processing robots in the future.
ACTA POLYTECHNICA HUNGARICA
(2022)
Article
Computer Science, Interdisciplinary Applications
Shenglin Wang, Jingqiong Zhang, Peng Wang, James Law, Radu Calinescu, Lyudmila Mihaylova
Summary: In Industry 5.0, Digital Twins provide flexibility and efficiency for smart manufacturing. Deep learning techniques are used to enhance the Digital Twin framework, enabling the detection and classification of human operators and robots during the manufacturing process. The framework shows promising results in accurately detecting and classifying actions of human operators and robots in various scenarios.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Yi Liu, Junpeng Qiu, Jincheng Wang, Junhe Lian, Zeran Hou, Junying Min
Summary: In this study, a double-sided robotic roller forming process was developed to form ultrahigh strength steels to thin-walled profiles. Synchronized laser heating and iterative path compensation method were used to reduce forming forces and achieve high-precision forming.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Zequn Zhang, Yuchen Ji, Dunbing Tang, Jie Chen, Changchun Liu
Summary: This paper proposes a digital twin system for human-robot collaboration (HRC) that overcomes the limitations of current methods and improves the overall performance. The system includes a human mesh recovery algorithm and uncertainty estimation to enhance the system's capabilities. Experimental results demonstrate the superiority of the proposed methods over baseline methods. The feasibility and effectiveness of the HRC system are validated through a case study involving component assembly.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Junmin Park, Taehoon Kim, Chengyan Gu, Yun Kang, Joono Cheong
Summary: This paper proposes a highly reliable and accurate collision estimator for robot manipulators in human-robot collaborative environments using the Bayesian approach. By assuming robot collisions as dynamic Markov processes, the estimator can integrate prior beliefs and measurements to produce current beliefs in a recursive form. The method achieves compelling performance in collision estimation with high accuracy and no false alarms.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Meng Wang, Kaixuan Chen, Panfeng Wang, Yimin Song, Tao Sun
Summary: In this study, a novel teleoperation machining mode and control strategy were proposed to improve efficiency and accuracy in small batch production of large casting parts. By using variable motion mapping and elastic compensation, constant cutting force was achieved, and the workpiece was protected by employing forbidden virtual fixtures and movement constraints on the slave robot.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Zhaoyu Li, Dong He, Xiangyu Li, Xiaoke Deng, Pengcheng Hu, Jiancheng Hao, Yue Hou, Hongyu Yu, Kai Tang
Summary: This paper presents a novel algorithm for planning a five-axis inspection path for arbitrary freeform surfaces. By converting the inspection path planning problem into a set-covering problem, the algorithm generates a near-minimum set of inspection paths that satisfy necessary constraints. Both computer simulation and physical inspection experiments confirm the effectiveness and advantages of the proposed method.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Hang Zhang, Wenhu Wang, Shusheng Zhang, Yajun Zhang, Jingtao Zhou, Zhen Wang, Bo Huang, Rui Huang
Summary: This paper introduces a novel framework based on deep reinforcement learning for generating machining process routes for designated parts. The framework utilizes graph representations of parts and employs convolutional graph neural networks for effective processing. Experimental results demonstrate the ability of the proposed method to generate efficient machining process routes and overcome limitations of traditional methods.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Regina Kyung-Jin Lee, Hao Zheng, Yuqian Lu
Summary: Future manufacturing will witness a shift towards collaboration and compassion in human-robot relationships. To enable seamless knowledge transfer, a unified knowledge representation system that can be shared by humans and robots is essential. The Human-Robot Shared Assembly Taxonomy (HR-SAT) proposed in this study allows comprehensive assembly tasks to be represented as a knowledge graph that is understandable by both humans and robots. HR-SAT incorporates rich assembly information and has diverse applications in process planning, quality checking, and human-robot collaboration.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Jianhui He, Lefeng Gu, Guilin Yang, Yiyang Feng, Silu Chen, Zaojun Fang
Summary: This paper presents a new modular kinematic error model for collaborative robots and proposes a portable self-calibration device to improve their positioning accuracy.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Hongwei Sun, Jixiang Yang, Han Ding
Summary: This paper proposes an asymmetrical FIR filter-based tool path smoothing algorithm to fully utilize the joint drive capability of robot manipulators. The algorithm considers the pose-dependent dynamics and constraints of the robot and improves motion efficiency by over 10% compared to traditional methods.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Dongsheng Ge, Huan Zhao, Yiwei Wang, Dianxi Li, Xiangfei Li, Han Ding
Summary: This paper focuses on learning a stable force control policy from human demonstration during contact transients. Based on the analysis of human demonstration data, a novel human-inspired force control strategy called compliant dynamical system (CDS) is proposed. The effectiveness of the proposed method is validated through simulation and real-world experiments.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Xuepeng Huang, Zhenzhong Wang, Lucheng Li, Qi Luo
Summary: This study models the stiffness of a robot and modifies the tool influence function (TIF) with the Preston equation in order to achieve uniform surface quality in robotic bonnet polishing (RBP) of optical components. Experimental results validate the accuracy of the modified model.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Mario D. Fiore, Felix Allmendinger, Ciro Natale
Summary: This paper presents a constraint-based programming framework for task specification and motion optimization. The framework can handle constraints on robot joint and Cartesian coordinates, as well as time dependency. It also compares with existing methods and provides numerical support through illustrative examples and case studies.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Yongxue Chen, Yaoan Lu, Ye Ding
Summary: This paper presents an optimization method for directly generating a six-degree-of-freedom toolpath for robotic flank milling. By optimizing the smoothness of the toolpath and the stiffness of the robot, the efficiency, accuracy, and finish of the machining are improved.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
Article
Computer Science, Interdisciplinary Applications
Chungang Zhuang, Haoyu Wang, Han Ding
Summary: This article proposes an end-to-end pipeline for synchronously regressing potential object poses from an unsegmented point cloud. It extracts point pair features and uses a voting architecture for instance feature extraction, along with a 3D heatmap for clustering votes and generating center seeds. An attention voting module is also employed to adaptively fuse point-wise features into instance-wise features. The network demonstrates robustness and improved performance in pose estimation.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2024)
