4.6 Article

Spiking Neural Networks for Visual Place Recognition Via Weighted Neuronal Assignments

Journal

IEEE ROBOTICS AND AUTOMATION LETTERS
Volume 7, Issue 2, Pages 4094-4101

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/LRA.2022.3149030

Keywords

Neurons; Robots; Task analysis; Visualization; Neuromorphics; Navigation; Biological neural networks; Neurorobotics; localization; autonomous vehicle navigation

Categories

Funding

  1. Australian Government [AUSMURIB000001]
  2. ONR MURI [N00014-19-1-2571]
  3. Intel Research [RV3.248]
  4. Queensland University of Technology (QUT) through the Centre for Robotics

Ask authors/readers for more resources

SNNs offer energy efficiency, low latency, and have potential in template matching and image recognition. This research introduces a high-performance SNN for VPR task, utilizing a novel assignment scheme to implement ambiguity-informed salience.
Spiking neural networks (SNNs) offer both compelling potential advantages, including energy efficiency and low latencies and challenges including the non-differentiable nature of event spikes. Much of the initial research in this area has converted deep neural networks to equivalent SNNs, but this conversion approach potentially negates some of the advantages of SNN-based approaches developed from scratch. One promising area for high-performance SNNs is template matching and image recognition. This research introduces the first high-performance SNN for the Visual Place Recognition (VPR) task: given a query image, the SNN has to find the closest match out of a list of reference images. At the core of this new system is a novel assignment scheme that implements a form of ambiguity-informed salience, by up-weighting single-place-encoding neurons and down-weighting ambiguous neurons that respond to multiple different reference places. In a range of experiments on the challenging Nordland, Oxford RobotCar, SPEDTest, Synthia, and St Lucia datasets, we show that our SNN achieves comparable VPR performance to state-of-the-art and classical techniques, and degrades gracefully in performance with an increasing number of reference places. Our results provide a significant milestone towards SNNs that can provide robust, energy-efficient, and low latency robot localization.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.6
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Engineering, Civil

HammerDrive: A Task-Aware Driving Visual Attention Model

Pierluigi Vito Amadori, Tobias Fischer, Yiannis Demiris

Summary: HammerDrive is a novel architecture for task-aware visual attention prediction in driving, able to reliably infer the driver's current focus of attention in real-time. The results show that HammerDrive outperforms a comparable state-of-the-art deep learning model in tracking driving maneuvers with significant improvement.

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS (2022)

Article Robotics

MultiRes-NetVLAD: Augmenting Place Recognition Training With Low-Resolution Imagery

Ahmad Khaliq, Michael Milford, Sourav Garg

Summary: In this paper, the authors propose an improved NetVLAD representation learning method, which utilizes a low-resolution image pyramid encoding to obtain richer place representations. The resulting multi-resolution feature pyramid can be easily aggregated using the VLAD algorithm, eliminating the need for concatenating or summing multiple patches. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in global descriptor-based retrieval.

IEEE ROBOTICS AND AUTOMATION LETTERS (2022)

Article Robotics

Improving Road Segmentation in Challenging Domains Using Similar Place Priors

Connor Malone, Sourav Garg, Ming Xu, Thierry Peynot, Michael Milford

Summary: This research improves road segmentation based on similar places by eliminating limitations of traditional methods. By using Visual Place Recognition to find similar places and integrating a novel segmentation quality metric, the system achieves state-of-the-art road segmentation performance in multiple challenging scenarios, without the need for prior training or access to the same geographical locations.

IEEE ROBOTICS AND AUTOMATION LETTERS (2022)

Article Automation & Control Systems

A RoboStack Tutorial Using the Robot Operating System Alongside the Conda and Jupyter Data Science Ecosystems

Tobias Fischer, Wolf Vollprecht, Silvio Traversaro, Sean Yen, Carlos Herrero, Michael Milford

IEEE ROBOTICS & AUTOMATION MAGAZINE (2022)

Article Robotics

Binary Neural Networks for Memory-Efficient and Effective Visual Place Recognition in Changing Environments

Bruno Ferrarini, Michael J. Milford, Klaus D. McDonald-Maier, Shoaib Ehsan

Summary: This article proposes a new class of highly compact models for visual place recognition (VPR) on resource-constrained platforms. By decreasing the precision of model parameters, reducing network depth, and decreasing the number of neurons in the classifier stage, the proposed models significantly reduce memory requirements and computational effort while maintaining state-of-the-art VPR performance.

IEEE TRANSACTIONS ON ROBOTICS (2022)

Article Robotics

Point Label Aware Superpixels for Multi-Species Segmentation of Underwater Imagery

Scarlett Raine, Ross Marchant, Brano Kusy, Frederic Maire, Tobias Fischer

Summary: Monitoring coral reefs using underwater vehicles and automated semantic segmentation can improve the range and availability of marine surveys. This study proposes a point label aware method for generating single-species segments in coral images, outperforming previous approaches and reducing computation time. The trained DeepLabv3+ model achieves impressive results in semantic segmentation on the UCSD Mosaics and Eilat datasets.

IEEE ROBOTICS AND AUTOMATION LETTERS (2022)

Article Communication

Fans not Customers! Kategoria in the Rise and Demise of the European Super League

Mike Milford

Summary: In 2021, the creation and subsequent demise of the Euro Super League highlighted the power of collective rhetoric, particularly the use of kategoria as a form of accusation against the league's owners. The supporters of the six English Premier League clubs positioned themselves as moral authorities, criticizing the owners for their greed and disconnect from the spirit of English soccer. This case demonstrates the intersection of community, morality, and simplified moral issues in contemporary sports.

COMMUNICATION & SPORT (2023)

Article Hospitality, Leisure, Sport & Tourism

Bannister in parliament: sport, synecdoche, and national identity

Mike Milford, Taylor J. Hendrickson

Summary: This research examines the role of Roger Bannister as a representative figure of the British identity and how parliament used him to shape and manage the country's image. The findings reveal the idealization of Bannister as a symbol of Britain's excellence and emphasize the need for careful management of his status. Athletes like Bannister are seen as instrumental in serving national purposes of identity construction and representation.

SPORT IN HISTORY (2023)

Article Robotics

How Many Events Do You Need? Event-Based Visual Place Recognition Using Sparse But Varying Pixels

Tobias Fischer, Michael Milford

Summary: Event cameras are of interest due to their characteristics such as high dynamic range, low latency, minimal motion blur, and energy efficiency. This letter explores the distinctiveness of event streams from a small subset of pixels for visual place recognition, demonstrating effectiveness in matching query observations to reference places.

IEEE ROBOTICS AND AUTOMATION LETTERS (2022)

Article Robotics

Improving Worst Case Visual Localization Coverage via Place-Specific Sub-Selection in Multi-Camera Systems

Stephen Hausler, Ming Xu, Sourav Garg, Punarjay Chakravarty, Shubham Shrivastava, Ankit Vora, Michael Milford

Summary: 6-DoF visual localization systems use 3D geometry to estimate camera pose accurately. By using hierarchical pipelines and learned 2D feature extractors, scalability and performance can be improved. This study investigates the usage of place-specific configurations to enhance worst-case localization performance, and demonstrates its effectiveness on the Ford AV benchmark dataset.

IEEE ROBOTICS AND AUTOMATION LETTERS (2022)

Article Robotics

Bayesian controller fusion: Leveraging control priors in deep reinforcement learning for robotics

Krishan Rana, Vibhavari Dasagi, Jesse Haviland, Ben Talbot, Michael Milford, Niko Sunderhauf

Summary: BCF is a hybrid control strategy that combines traditional hand-crafted controllers with model-free deep reinforcement learning. By fusing uncertainty-aware distributional outputs, BCF arbitrates control between the two systems and achieves better performance. It accelerates learning and ensures safe exploration and deployment.

INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH (2023)

Proceedings Paper Computer Science, Artificial Intelligence

Hyperdimensional Feature Fusion for Out-of-Distribution Detection

Samuel Wilson, Tobias Fischer, Niko Sunderhauf, Feras Dayoub

Summary: We apply ideas from Hyperdimensional Computing to improve Out-of-Distribution (OOD) detection. Instead of using a single layer, our method uses semi-orthogonal projection matrices to project feature maps from multiple layers into a shared vector space, creating expressive descriptor vectors for all classes. By calculating cosine similarity, our method consistently identifies OOD samples with competitive performance and high efficiency. Our method can be combined with existing OOD detectors for further improvement.

2023 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV) (2023)

Article Automation & Control Systems

A RoboStack Tutorial

Tobias Fischer, Wolf Vollprecht, Silvio Traversaro, Sean Yen, Carlos Herrero, Michael Milford

IEEE ROBOTICS & AUTOMATION MAGAZINE (2022)

No Data Available