4.6 Article

Active Perception for Foreground Segmentation: An RGB-D Data-Based Background Modeling Method

Journal

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TASE.2019.2893414

Keywords

Active perception; Image color analysis; Sensors; Computer vision; Image segmentation; Active perception; background modeling; foreground segmentation; RGB-D camera

Funding

  1. Shenzhen Science and Technology Innovation Project [JCYJ20160428154842603, JCYJ20170413161616163]
  2. Hong Kong Research Grant Council (RGC) [11210017, 16212815, 21202816, 14205914, 14200618]
  3. ITC ITF Project [ITS/236/15]
  4. National Natural Science Foundation of China [U1713211]

Ask authors/readers for more resources

Foreground moving object segmentation is a fundamental problem in many computer vision applications. As a solution for foreground segmentation, background modeling has been intensively studied over past years and many effective algorithms have been developed. However, accurate foreground segmentation is still a difficult problem. Currently, most of the algorithms work solely within the color space, in which the segmentation performance is prone to be degraded by a multitude of challenges, such as illumination changes, shadows, automatic camera adjustments, and color camouflage. RGB-D cameras are active visual sensors that provide depth measurements along with color images. We present in this paper an innovative background modeling method by using both the color and depth information from an RGB-D camera. The proposed method is evaluated using a public RGB-D data set. Various experiments confirm that our method is able to achieve superior performance compared with existing well-known methods. Note to Practitioners-This paper investigates background modeling for foreground segmentation with active perception. Recent RGB-D cameras that leverage the active perception technology have advanced many computer vision algorithms. In this paper, we develop a background modeling method to achieve superior performance by using an RGB-D camera instead of a color camera. Due to the use of the active sensing technology, the proposed method is characterized by its robustness to common challenges. Our method could be used for improving existing infrastructures, such as visual surveillance systems for parking spaces. Moreover, the simple design of our method allows it to be easily deployed on various computing platforms, which facilitates many practical applications that usually require embedded computing devices. However, our method cannot run real timely at the current status. We believe that it can be further improved using parallel programming techniques to meet the real-time requirement.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.6
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Automation & Control Systems

Motion Hysteresis Compensation Based on Motor Current Segmentation for Elongated Cable-Driven Surgical Instruments

Yongchen Guo, Bo Pan, Yanwen Sun, Guojun Niu, Yili Fu, Max Q. -H. Meng

Summary: This paper proposes a motion hysteresis compensation method for cable-driven mechanism, which leverages the relationship between actuate motor current and hysteresis phases to generate compensation curves. The method is shown to improve the accuracy and efficiency of compensation, and has potential applications in large-scale identification or repeated usage scenarios.

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING (2023)

Article Automation & Control Systems

Generalized n-Dimensional Rigid Registration: Theory and Applications

Jin Wu, Miaomiao Wang, Hassen Fourati, Hui Li, Yilong Zhu, Chengxi Zhang, Yi Jiang, Xiangcheng Hu, Ming Liu

Summary: This study investigates the generalized rigid registration problem in high-dimensional Euclidean spaces and proposes a method to minimize the loss function using the Cayley formula. By deriving a closed-form linear least-square solution, the registration covariances are obtained, providing accurate probabilistic descriptions. The proposed method demonstrates its efficiency in terms of computation time and accuracy compared to previous algorithms. Additionally, it is applied to an interpolation problem on a special Euclidean group and shows practical superiority in covariance-aided Lidar mapping for robotic navigation.

IEEE TRANSACTIONS ON CYBERNETICS (2023)

Article Robotics

Real-Time Neural Dense Elevation Mapping for Urban Terrain With Uncertainty Estimations

Bowen Yang, Qingwen Zhang, Ruoyu Geng, Lujia Wang, Ming Liu

Summary: Having good knowledge of terrain information is essential for improving the performance of legged robots in locomotion and navigation on complex terrains. We present a novel framework that generates dense robot-centric elevation maps online from sparse LiDAR observations, and provides uncertainty estimations. Our approach ensures high robustness and computational efficiency by using a novel pre-processing and point features representation approach. The generative Bayesian model recovers detailed terrain structures and provides pixel-wise reconstruction uncertainty, benefiting the downstream tasks of legged robots.

IEEE ROBOTICS AND AUTOMATION LETTERS (2023)

Article Automation & Control Systems

Human-Aware Path Planning With Improved Virtual Doppler Method in Highly Dynamic Environments

Kuanqi Cai, Weinan Chen, Chaoqun Wang, Shuang Song, Max Q. -H. Meng

Summary: In this paper, an integrated framework for finding the optimal path in complex environments is proposed, considering collision risk, social norms, and crowded areas. The framework includes a dynamic group model, a collision risk and human space model, and an improved navigation method. Experimental results demonstrate that this method can generate the optimal human-aware collision-free path in complex environments.

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING (2023)

Article Automation & Control Systems

Motion Planning of Manipulator by Points-Guided Sampling Network

Erli Lyu, Tingting Liu, Jiaole Wang, Shuang Song, Max Q-H Meng

Summary: This paper proposes a points-guided sampling net (PGSN) that utilizes geometric information to guide the sampling process in a sampling-based motion planner. By extracting geometric features from point clouds, a VAE feature extraction net and a multi-modal sampling net are designed. Additionally, a sampling-based motion planning algorithm called PG-RRT is presented based on PGSN, and its effectiveness is demonstrated through experiments.

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING (2023)

Article Chemistry, Analytical

A Novel Real-Time Autonomous Crack Inspection System Based on Unmanned Aerial Vehicles

Kwai-Wa Tse, Rendong Pi, Yuxiang Sun, Chih-Yung Wen, Yurong Feng

Summary: Traditional methods for crack inspection in large infrastructures are time-consuming and costly as they require multiple devices and instruments. In this study, we propose a real-time crack inspection system based on unmanned aerial vehicles, which successfully detects and classifies various types of cracks. The system accurately identifies the crack positions in the world coordinate system. Our detector, an improved YOLOv4 with an attention module, achieves 90.02% mean average precision (mAP) and outperforms the YOLOv4-original by 5.23% in terms of mAP. The proposed system is low-cost, lightweight, and not restricted by navigation trajectories. Experimental results demonstrate its robustness and effectiveness in real-world crack inspection tasks.

SENSORS (2023)

Article Robotics

Expanding Sparse LiDAR Depth and Guiding Stereo Matching for Robust Dense Depth Estimation

Zhenyu Xu, Yuehua Li, Shiqiang Zhu, Yuxiang Sun

Summary: Dense depth estimation is crucial for various applications and stereo matching has become popular for this task. However, stereo-LiDAR fusion is a promising method to overcome challenges like low textures and occlusions. To address the issue of sparse and uneven distribution of LiDAR data, a semi-dense depth expansion method is proposed using RGB images as a reference. This method shows superior performance over traditional sparse invariant convolution methods in terms of accuracy and robustness, as proven by experimental results on different datasets.

IEEE ROBOTICS AND AUTOMATION LETTERS (2023)

Article Robotics

RNGDet++: Road Network Graph Detection by Transformer With Instance Segmentation and Multi-Scale Features Enhancement

Zhenhua Xu, Yuxuan Liu, Yuxiang Sun, Ming Liu, Lujia Wang

Summary: The road network graph plays a critical role in downstream tasks of autonomous driving. To annotate road network graphs effectively and efficiently, automatic algorithms are demanded. However, existing methods suffer from hard-coded algorithms and poor performance. We propose RNGDet++, an improved method with an instance segmentation head and the ability to leverage multi-scale features, which outperforms baseline methods on large-scale public datasets.

IEEE ROBOTICS AND AUTOMATION LETTERS (2023)

Article Robotics

CEKD: Cross-Modal Edge-Privileged Knowledge Distillation for Semantic Scene Understanding Using Only Thermal Images

Zhen Feng, Yanning Guo, Yuxiang Sun

Summary: Semantic scene understanding using thermal images is challenging due to the lack of color information and blurred edges. To tackle this, we propose a cross-modal edge-privileged knowledge distillation framework that utilizes a fusion-based segmentation network as a teacher to guide a student network that only uses thermal images. Experimental results demonstrate that the student network achieves superior performance with thermal images under the guidance of the teacher.

IEEE ROBOTICS AND AUTOMATION LETTERS (2023)

Article Automation & Control Systems

Full-Coverage Path Planning and Stable Interaction Control for Automated Robotic Breast Ultrasound Scanning

Ziwen Wang, Baoliang Zhao, Peng Zhang, Liang Yao, Qiong Wang, Bing Li, Max Q. -H. Meng, Ying Hu

Summary: A robotic system for automated breast ultrasound scanning is proposed in this study. The system obtains the point cloud of the breast from multiple angles and registers them together for accurate tissue shape reconstruction. Scan path planning is performed using an isometric 3-D point cloud searching algorithm for full and uniform coverage. A contact force-strain regression model is built for tissue deformation estimation and used to correct the planned scanning path. Additionally, a probe-tissue interaction model with scanning resistance is built for normal probe orientation adjustment.

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS (2023)

Article Robotics

Closed-Loop Magnetic Manipulation for Robotic Transesophageal Echocardiography

Keyu Li, Yangxin Xu, Ziqi Zhao, Ang Li, Max Q. -H. Meng

Summary: This paper presents a closed-loop magnetic manipulation framework for robotic transesophageal echocardiography (TEE) acquisitions. The framework utilizes magnetic control methods for more direct, intuitive, and accurate manipulation of the probe. Extensive experiments validate the effectiveness of the framework in terms of localization accuracy, update rate, workspace size, and tracking accuracy.

IEEE TRANSACTIONS ON ROBOTICS (2023)

Article Biotechnology & Applied Microbiology

Finite element study of sagittal fracture location on thoracolumbar fracture treatment

Xilong Cui, Junjun Zhu, Wanmei Yang, Yuxiang Sun, Xiuling Huang, Xiumei Wang, Haiyang Yu, Chengmin Liang, Zikai Hua

Summary: This study aimed to investigate the effects of the sagittal location of the fracture region on the biomechanics of the internal fixation system and surgical strategy. The results showed that sagittal location has an impact on stress distribution of the fixation system, but does not influence the selection of surgical strategy.

FRONTIERS IN BIOENGINEERING AND BIOTECHNOLOGY (2023)

Article Automation & Control Systems

Learning-Based Risk-Bounded Path Planning Under Environmental Uncertainty

Fei Meng, Liangliang Chen, Han Ma, Jiankun Wang, Max Q. -H. Meng

Summary: Building a general and efficient path planning framework in uncertain nonconvex environments is challenging. Traditional methods involve convexifying obstacles and presuming Gaussian distribution, which are not universal. Our novel neural risk-bounded path planner quickly finds near-optimal solutions with acceptable collision probability in complex environments.

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING (2023)

Article Robotics

ACAM-FoC: A Deep Neural Network Augmented From CAM-FoC to Measure the Grip Force of Mass-Produced Elongated Surgical Instruments

Yongchen Guo, Bo Pan, Yili Fu, Max Q. -H. Meng

Summary: Learning-based grip force measurement methods in RAMIS outperform traditional model-based methods and avoid the issues of sensor-based approaches. However, there is limited research on grip force measurement in mass-produced surgical instruments. This letter proposes a novel learning-based method, ACAM-FoC, which considers the differences in motion hysteresis and mechanism friction among mass-produced surgical instruments.

IEEE ROBOTICS AND AUTOMATION LETTERS (2023)

Review Engineering, Biomedical

3-D Rigid Point Set Registration for Computer-Assisted Orthopedic Surgery (CAOS): A Review From the Algorithmic Perspective

Zhe Min, Ang Zhang, Zhengyan Zhang, Jiaole Wang, Shuang Song, Hongliang Ren, Max Q. -H. Meng

Summary: This paper provides a concise review of rigid point set registration methods in computer-assisted orthopedic surgery (CAOS). The challenge lies in establishing point correspondences between two point sets under noise, outliers, and partial overlapping. The paper discusses and compares the advantages and disadvantages of surveyed registration algorithms, and also proposes potential future research directions.

IEEE TRANSACTIONS ON MEDICAL ROBOTICS AND BIONICS (2023)

No Data Available