 To improve the efficiency of deep reinforcement learning (DRL) based methods for robot manipulator trajectory planning in random working environment. Different from the traditional sparse reward function , we present three dense reward functions in this paper. Firstly, posture reward function is proposed to accelerate the learning process with a more reasonable trajectory by modeling the distance and direction constraints, which can reduce the blindness of exploration. Secondly, to improve the stability, a reward function at stride reward is proposed by modeling the distance and movement distance of joints constraints, it can make the learning process more stable. In order to further improve learning efficiency, we are inspired by the cognitive process of human behavior and propose a stage incentive mechanism, including  hard stage incentive reward function and  soft stage incentive reward function. Extensive experiments show that the soft stage incentive reward function proposed is able to improve  convergence rate by up to 46.9\% with the state-of-the-art DRL methods. The percentage increase in  convergence mean reward is 4.4 \%~15.5\% and the percentage decreases with respect to standard deviation by 21.9 \%~63.2\% . In the evaluation  , the success rate of trajectory planning for robot manipulator is up to 99.6\%.
(This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.) To improve the efficiency of deep reinforcement learning (DRL) -based methods for robot manipulator trajectory planning in random working environments , we present three dense reward functions . These rewards differ from the traditional sparse reward. First, a posture reward function is proposed to speed up the learning process with a more reasonable trajectory by modeling the distance and direction constraints, which can reduce the blindness of exploration. Second, a stride reward function is proposed to improve the stability of the learning process by modeling the distance and movement distance of joint constraints. Finally, in order to further improve learning efficiency, we are inspired by the cognitive process of human behavior and propose a stage incentive mechanism, including a hard stage incentive reward function and a soft stage incentive reward function. Extensive experiments show that the soft stage incentive reward function is able to improve the convergence rate by up to 46.9\% with the state-of-the-art DRL methods. The percentage increase in the convergence mean reward was 4.4 -15.5\% and the percentage decreases with respect to standard deviation were 21.9 -63.2\% . In the evaluation experiments , the success rate of trajectory planning for a robot manipulator reached 99.6\%.
A
(This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.)
R
based
-based
R
environment. Different from the traditional sparse reward function
environments
R
in this paper. Firstly,
. These rewards differ from the traditional sparse reward. First, a
R
accelerate
speed up
R
Secondly, to improve the stability, a reward function at stride reward is proposed
Second, a stride reward function is proposed to improve the stability of the learning process
R
joints constraints, it can make the learning process more stable. In
joint constraints. Finally, in
A
a
A
a
D
proposed
A
the
A
the
R
is
was
R
\%~15.5\%
-15.5\%
R
by
were
R
\%~63.2\%
-63.2\%
A
experiments
R
robot manipulator is up to
a robot manipulator reached