  Limb deficiency severely affects the daily lives of amputees and drives
efforts to provide functional robotic prosthetic hands to compensate this
deprivation. Convolutional neural network-based computer vision control of the
prosthetic hand has received increased attention as a method to replace or
complement physiological signals due to its reliability by training visual
information to predict the hand gesture. Mounting a camera into the palm of a
prosthetic hand is proved to be a promising approach to collect visual data.
However, the grasp type labelled from the eye and hand perspective may differ
as object shapes are not always symmetric. Thus, to represent this difference
in a realistic way, we employed a dataset containing synchronous images from
eye- and hand- view, where the hand-perspective images are used for training
while the eye-view images are only for manual labelling. Electromyogram (EMG)
activity and movement kinematics data from the upper arm are also collected for
multi-modal information fusion in future work. Moreover, in order to include
human-in-the-loop control and combine the computer vision with physiological
signal inputs, instead of making absolute positive or negative predictions, we
build a novel probabilistic classifier according to the Plackett-Luce model. To
predict the probability distribution over grasps, we exploit the statistical
model over label rankings to solve the permutation domain problems via a
maximum likelihood estimation, utilizing the manually ranked lists of grasps as
a new form of label. We indicate that the proposed model is applicable to the
most popular and productive convolutional neural network frameworks.

Computer Vision and Pattern Recognition;Artificial Intelligence