Article
Robotics
Philipp Foehn, Elia Kaufmann, Angel Romero, Robert Penicka, Sihao Sun, Leonard Bauersfeld, Thomas Laengle, Giovanni Cioffi, Yunlong Song, Antonio Loquercio, Davide Scaramuzza
Summary: Agilicious is a hardware and software framework designed for autonomous, agile quadrotor flight, supporting both model-based and neural network-based controllers. It offers a combination of high-performance hardware and flexible software stack, making it suitable for various tasks and environments, as well as hardware-in-the-loop simulation.
Article
Automation & Control Systems
Jian Di, Shaofeng Chen, Pengfei Li, Xinghu Wang, Haibo Ji, Yu Kang
Summary: This article introduces a cooperative-competitive strategy based on the leader-wingman framework for autonomous multidrone racing. By using a game-theoretic method to compete against opponents and a line of sight-based cooperative method to support the leader, the win rate can be improved with lower computational cost. The strategy is validated through extensive simulation and real-world comparisons with state-of-the-art approaches.
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS
(2023)
Article
Engineering, Civil
Umit Celik, Haluk Eren
Summary: As the number of UAVs and the market size have been expanding rapidly, projects such as NextGen and SESAR aim to include UAVs in air traffic for more effective management. The analysis of flight data offers important insights into UAV operations. This study aims to extract flight fingerprints using various machine learning techniques, reducing multidimensional UAV sensor data and comparing different manifold types and classification methods to achieve the highest classification accuracy.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2023)
Article
Robotics
Antonio Loquercio, Elia Kaufmann, Rene Ranftl, Matthias Mueller, Vladlen Koltun, Davide Scaramuzza
Summary: Quadrotors are agile machines that can navigate complex environments at high speeds, but traditional methods of navigation may lead to increased latency and error accumulation. A new end-to-end approach is proposed to autonomously fly quadrotors through challenging environments with purely onboard sensing and computation, outperforming traditional obstacle avoidance pipelines.
Article
Robotics
Angel Romero, Sihao Sun, Philipp Foehn, Davide Scaramuzza
Summary: In this article, we propose a Model Predictive Contouring Control (MPCC) method to tackle the problem of flying time-optimal trajectories through multiple waypoints with quadrotors. By solving the time allocation problem and the control problem concurrently, our MPCC can generate a path that approaches the true time-optimal one in real time.
IEEE TRANSACTIONS ON ROBOTICS
(2022)
Article
Robotics
Michael O'Connell, Guanya Shi, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, Soon-Jo Chung
Summary: This study focuses on the flight control problem of UAVs in dynamic high-speed winds and proposes a learning-based approach called Neural-Fly. By incorporating pretrained representations and using domain adversarially invariant meta-learning algorithm, Neural-Fly achieves precise flight control under challenging wind conditions and provides robustness guarantees.
Article
Computer Science, Artificial Intelligence
Hao-Yun Chen, Pei-Han Huang, Li-Chen Fu
Summary: This paper proposes a hierarchical path planning algorithm that combines RGB camera and LiDAR to capture local crowd movement and predict nearby people's movement. It generates appropriate global path for the robot using crowd information and social norms. The system accurately tracks human locations and allows the robot to plan efficient and socially acceptable paths.
Article
Robotics
Yunlong Song, Davide Scaramuzza
Summary: A novel framework of policy search for model predictive control is proposed in the study, utilizing policy search to automatically select high-level decision variables for MPC. The formulation of a parameterized controller allows optimizing policies in a self-supervised manner.
IEEE TRANSACTIONS ON ROBOTICS
(2022)
Article
Computer Science, Artificial Intelligence
Boban Sazdic-Jotic, Ivan Pokrajac, Jovan Bajcetic, Boban Bondzulic, Danilo Obradovic
Summary: The research indicates that the use of drones has significantly improved and expanded, and a potential deep learning algorithm has been proposed as an anti-drone solution. The results show that the proposed algorithm has great potential in detecting and identifying drones, and exhibits high accuracy in detecting multiple drones.
EXPERT SYSTEMS WITH APPLICATIONS
(2022)
Article
Engineering, Electrical & Electronic
Efstratios Kakaletsis, Ioannis Mademlis, Nikos Nikolaidis, Ioannis Pitas
Summary: This paper presents a centralized, vision-based method for robust, on-the-fly 3D localization and mapping of human crowds in large-scale outdoor environments. The method involves independently detecting crowds through multiple UAV camera feeds, converting 2D crowd heatmaps into a common 3D terrain/map, and fusing crowd heatmaps from different drones/cameras using Bayesian filtering.
SIGNAL PROCESSING-IMAGE COMMUNICATION
(2021)
Article
Robotics
Sihao Sun, Angel Romero, Philipp Foehn, Elia Kaufmann, Davide Scaramuzza
Summary: This article compares two state-of-the-art control frameworks for accurate trajectory-tracking control of quadrotors. The study evaluates the nonlinear-model-predictive controller (NMPC) and the differential-flatness-based controller (DFBC) through simulations and real-world experiments. The findings demonstrate that the NMPC performs better in tracking dynamically infeasible trajectories, but at the expense of longer computation time and the risk of numerical convergence issues. The experiments also highlight the necessity of using an inner loop controller and aerodynamic drag model for agile trajectory tracking.
IEEE TRANSACTIONS ON ROBOTICS
(2022)
Article
Automation & Control Systems
Yuanyang Zhu, Zhi Wang, Chunlin Chen, Daoyi Dong
Summary: This article introduces a rule-based reinforcement learning (RuRL) algorithm for efficient navigation. By employing a wall-following rule to generate a closed-loop trajectory, a reduction rule to shrink the trajectory, and the Pledge rule to guide the exploration strategy, RuRL achieves improved navigation performance in real robot navigation experiments.
IEEE-ASME TRANSACTIONS ON MECHATRONICS
(2022)
Article
Engineering, Civil
Vishal Mahajan, Emmanouil Barmpounakis, Md. Rakibul Alam, Nikolas Geroliminis, Constantinos Antoniou
Summary: Unmanned aerial systems, or drones, can collect traffic data and process raw data to remove noise and anomalies, making it suitable for developing traffic flow or crash risk models. This study uses a part of the pNEUMA dataset captured by drones in Athens, Greece, and applies smoothing filters and Extreme Gradient Boosting to process vehicle trajectories. This approach can help accelerate research in microscopic traffic analysis.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2023)
Article
Computer Science, Information Systems
Vignesh Sushrutha Raghavan, Dimitrios Kanoulas, Darwin G. Caldwell, Nikos G. Tsagarakis
Summary: This article discusses legged and wheeled locomotion methods used by robots for navigation, and the advantages of combining them to create a hybrid legged-wheeled locomotion. The article presents a review of a previously presented legged-wheeled footprint reconfiguring global planner, as well as new work on local obstacle pushing. The combination of these global and local planners forms a major part of the agile reconfigurable navigation suite for the legged-wheeled hybrid CENTAURO robot.
Article
Robotics
Ziyu Zhou, Gang Wang, Jian Sun, Jikai Wang, Jie Chen
Summary: This letter proposes a novel approach to computing time-optimal trajectories, which significantly accelerates trajectory planning by fixing nodes with waypoint constraints and adopting separate sampling intervals. Furthermore, the planned paths are tracked via a time-adaptive model predictive control scheme, enhancing tracking accuracy and robustness.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2023)
Article
Computer Science, Artificial Intelligence
Philipp Foehn, Dario Brescianini, Elia Kaufmann, Titus Cieslewski, Mathias Gehrig, Manasi Muglikar, Davide Scaramuzza
Summary: This paper presents a novel system for autonomous, vision-based drone racing, achieving second place at the 2019 AlphaPilot Challenge by combining learned data abstraction, nonlinear filtering, and time-optimal trajectory planning.
Article
Robotics
Nico Messikommer, Daniel Gehrig, Mathias Gehrig, Davide Scaramuzza
Summary: Reliable perception during fast motion maneuvers or in high dynamic range environments is crucial for robotic systems. Event cameras, due to their robustness to challenging conditions, have great potential to increase the reliability of robot vision. However, the scarcity of labeled datasets has hindered the development of event-based vision. To address this issue, the proposed task transfer method trains models directly with labeled images and unlabeled event data.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2022)
Article
Robotics
Drew Hanover, Philipp Foehn, Sihao Sun, Elia Kaufmann, Davide Scaramuzza
Summary: This paper proposes a novel hybrid adaptive nonlinear model predictive control (NMPC) method called L-1-NMPC, which learns and compensates for model uncertainties online to improve performance of quadrotor flight in challenging environments. The method demonstrates high flexibility and robustness, achieving significant tracking error reduction and improved adaptability in the presence of complex aerodynamic effects and unknown disturbances.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2022)
Article
Robotics
Giovanni Cioffi, Titus Cieslewski, Davide Scaramuzza
Summary: Robotic practitioners generally approach the vision-based SLAM problem through discrete-time formulations, but these formulations require tailored algorithms and simplifying assumptions in the presence of high-rate and/or asynchronous measurements. On the other hand, continuous-time SLAM allows the fusion of multiple sensor modalities in an intuitive fashion, but may result in worse trajectory estimates.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2022)
Article
Robotics
Leonard Bauersfeld, Davide Scaramuzza
Summary: This article introduces the importance of multicopters in various application areas, and proposes a method for estimating the range, endurance, and optimal speed of multicopters. The accuracy and feasibility of this method are validated through experiments and flights. The article also provides a pen-and-paper algorithm to assist future researchers in building drones with maximum range and endurance.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2022)
Article
Robotics
Fang Nan, Sihao Sun, Philipp Foehn, Davide Scaramuzza
Summary: This study proposes a fault-tolerant controller using nonlinear model predictive control (NMPC) to stabilize and control a quadrotor in the event of complete failure of a single rotor. Unlike existing methods, this approach considers the full nonlinear dynamics of the damaged quadrotor and the thrust constraint of each rotor.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2022)
Article
Robotics
Yunlong Song, Davide Scaramuzza
Summary: A novel framework of policy search for model predictive control is proposed in the study, utilizing policy search to automatically select high-level decision variables for MPC. The formulation of a parameterized controller allows optimizing policies in a self-supervised manner.
IEEE TRANSACTIONS ON ROBOTICS
(2022)
Article
Robotics
Lintong Zhang, Michael Helmberger, Lanke Frank Tarimo Fu, David Wisth, Marco Camurri, Davide Scaramuzza, Maurice Fallon
Summary: To drive the advancement of SLAM systems, we created the Hilti-Oxford Dataset, which includes various challenges to test the performance of SLAM algorithms in different scenarios. We implemented a novel ground truth collection method to accurately measure pose errors with millimeter accuracy. The dataset attracted a large number of researchers to participate in the Hilti SLAM challenge.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2023)
Article
Robotics
Tim Salzmann, Elia Kaufmann, Jon Arrizabalaga, Marco Pavone, Davide Scaramuzza, Markus Ryll
Summary: Model Predictive Control (MPC) is a popular framework in embedded control for high-performance autonomous systems. However, the use of accurate dynamics models is crucial for achieving good control performance. This study presents Real-time Neural MPC, a framework that efficiently integrates large neural network architectures as dynamics models within a model-predictive control pipeline. The experiments conducted in simulation and on a quadrotor platform show the capabilities of the system to run learned models with large modeling capacity using gradient-based online optimization MPC, resulting in significant improvements compared to state-of-the-art MPC approaches.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2023)
Article
Robotics
Giovanni Cioffi, Leonard Bauersfeld, Elia Kaufmann, Davide Scaramuzza
Summary: Inertial odometry is a promising solution for state estimation in agile quadrotor flight. However, relying solely on inertial measurements leads to drift in pose estimates. This study proposes a learning-based odometry algorithm using an IMU as the only sensor for autonomous drone racing. The algorithm combines a model-based filter driven by inertial measurements with a learning-based module that uses thrust measurements. The results show the superiority of this inertial odometry algorithm in pose estimation compared to other visual-inertial odometry methods, indicating its potential in agile quadrotor flight.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2023)
Proceedings Paper
Automation & Control Systems
Charith Munasinghe, Fatemeh Mohammadi Amin, Davide Scaramuzza, Hans Wernher van de Venn
Summary: Safe human-robot collaboration is crucial in the emerging Industry 5.0 paradigm, where conventional robots are being replaced by more intelligent and flexible collaborative robots. However, the lack of research and dedicated datasets for 3D semantic segmentation of collaborative robot workspaces hinders safe and efficient collaboration. This work addresses this limitation by developing a new dataset named COVERED and benchmarking state-of-the-art algorithm performance for real-time semantic segmentation.
2022 IEEE 27TH INTERNATIONAL CONFERENCE ON EMERGING TECHNOLOGIES AND FACTORY AUTOMATION (ETFA)
(2022)
Proceedings Paper
Automation & Control Systems
Maryam Rezayati, Grammatiki Zanni, Ying Zaoshi, Davide Scaramuzza, Hans Wernher van de Venn
Summary: Direct physical interaction with robots is important in flexible production scenarios, but poses risks to operators. Simple measures can prevent injuries, but hinder true human-robot cooperation. More sophisticated solutions are needed in human-robot collaboration scenarios.
2022 IEEE 27TH INTERNATIONAL CONFERENCE ON EMERGING TECHNOLOGIES AND FACTORY AUTOMATION (ETFA)
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Zhaoning Sun, Nico Messikommer, Daniel Gehrig, Davide Scaramuzza
Summary: This research introduces ESS (Event-based Semantic Segmentation), a method that transfers the semantic segmentation task from labeled image datasets to unlabeled events through unsupervised domain adaptation (UDA). Compared to existing methods, our approach aligns event embeddings with image embeddings without the need for video data or per-pixel alignment between images and events, and without the need to infer motion from still images. We also introduce DSEC-Semantic, a large-scale event-based dataset with fine-grained labels. Experimental results show that ESS outperforms existing UDA approaches using image labels alone and even surpasses state-of-the-art supervised approaches when combined with event labels in both DDD17 and DSEC-Semantic datasets. ESS is a general-purpose method that opens up new and previously inaccessible research directions for event cameras.
COMPUTER VISION, ECCV 2022, PT XXXIV
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Stepan Tulyakov, Alfredo Bochicchio, Daniel Gehrig, Stamatios Georgoulis, Yuanyou Li, Davide Scaramuzza
Summary: This study addresses several issues in video frame interpolation, including unstable fusion process of complementary interpolation results, inefficient motion estimation process, and potential artifacts in low contrast regions. Additionally, a large-scale dataset with challenging scenes is constructed, and the reconstruction quality is improved through multi-scale feature-level fusion and one-shot non-linear inter-frame motion computation.
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022)
(2022)
