Object recognition combining vision and touch


Corradi, T., Hall, P. and Iravani, P., 2017. Object recognition combining vision and touch. Robotics and Biomimetics, 4 (2).

Related documents:

This repository does not currently have the full-text of this item.
You may be able to access a copy if URLs are provided below. (Contact Author)

Official URL:

Related URLs:


This paper explores ways of combining vision and touch for the purpose of object recognition. In particular, it focuses on scenarios when there are few tactile training samples (as these are usually costly to obtain) and when vision is artificially impaired. Whilst machine vision is a widely studied field, and machine touch has received some attention recently, the fusion of both modalities remains a relatively unexplored area. It has been suggested that, in the human brain, there exist shared multi-sensorial representations of objects. This provides robustness when one or more senses are absent or unreliable. Modern robotics systems can benefit from multi-sensorial input, in particular in contexts where one or more of the sensors perform poorly. In this paper, a recently proposed tactile recognition model was extended by integrating a simple vision system in three different ways: vector concatenation (vision feature vector and tactile feature vector), object label posterior averaging and object label posterior product. A comparison is drawn in terms of overall accuracy of recognition and in terms of how quickly (number of training samples) learning occurs. The conclusions reached are: (1) the most accurate system is “posterior product”, (2) multi-modal recognition has higher accuracy to either modality alone if all visual and tactile training data are pooled together, and (3) in the case of visual impairment, multi-modal recognition “learns faster”, i.e. requires fewer training samples to achieve the same accuracy as either other modality.


Item Type Articles
CreatorsCorradi, T., Hall, P. and Iravani, P.
Related URLs
URLURL Type Full-text
DepartmentsFaculty of Engineering & Design > Mechanical Engineering
Faculty of Science > Computer Science
Research CentresCentre for Power Transmission & Motion Control
Centre for Sustainable Chemical Technologies
ID Code55480


Actions (login required)

View Item