Human driving behavior modeling is of great importance for designing safe, smart, smooth as well as personalized autonomous driving systems. In this paper, an internal reward function-based driving model that emulates the human's internal decision-making mechanism is proposed. Besides, a sampling-based inverse reinforcement learning (IRL) algorithm that learns the reward function from human naturalistic driving data is also developed. A polynomial trajectory sampler is adopted to generate feasible trajectories and approximate the partition function in the maximum entropy IRL framework, and a dynamic and interactive environment is built upon the static driving dataset to estimate the generated trajectories considering the mutual dependency of agents' actions . The proposed method is applied to learn personalized reward functions for individual human drivers from the NGSIM  dataset. The qualitative results demonstrate that the learned reward function is able to  interpret their decisions. The quantitative results also reveal that the  personalized modeling method significantly outperforms the general modeling approach, reducing the  errors in human likeness by 24 \%, and the proposed reward-function-based method delivers better results compared to other baseline methods . Moreover, it is found that estimating the response actions of surrounding vehicles plays an integral role in estimating the trajectory accurately and achieving a better generalization ability .
Driving behavior modeling is of great importance for designing safe, smart, and personalized autonomous driving systems. In this paper, an internal reward function-based driving model that emulates the human's internal decision-making mechanism is utilized. To infer the reward function from naturalistic human driving data, we propose a structural assumption about human driving behavior that focuses on discrete latent driving intentions. It converts the continuous behavior modeling problem to a discrete setting and thus makes maximum entropy inverse reinforcement learning (IRL) tractable to learn reward functions. Specifically, a polynomial trajectory sampler is adopted to generate candidate trajectories considering high-level intentions and approximate the partition function in the maximum entropy IRL framework, and an environment model considering interactive behaviors among the ego and surrounding vehicles is built to better estimate the generated trajectories . The proposed method is applied to learn personalized reward functions for individual human drivers from the NGSIM highway dataset. The qualitative results demonstrate that the learned reward function is able to explicitly express the preferences of different drivers and interpret their decisions. The quantitative results reveal that the learned reward function is robust, which is manifested by only a marginal decline in proximity to the human driving trajectories when applying the reward function in the testing conditions. For the testing performance, the personalized modeling method outperforms the general modeling approach, reducing the modeling errors in human likeness (a custom metric to gauge accuracy) by 23 \%, and these two methods deliver better results compared to other baseline methods .
R
Human driving
Driving
R
smooth as well as
and
R
proposed. Besides, a sampling-based
utilized. To infer the reward function from naturalistic human driving data, we propose a structural assumption about human driving behavior that focuses on discrete latent driving intentions. It converts the continuous behavior modeling problem to a discrete setting and thus makes maximum entropy
R
algorithm that learns the reward function from human naturalistic driving data is also developed. A
tractable to learn reward functions. Specifically, a
R
feasible trajectories
candidate trajectories considering high-level intentions
R
a dynamic and interactive environment is built upon the static driving dataset to
an environment model considering interactive behaviors among the ego and surrounding vehicles is built to better
D
considering the mutual dependency of agents' actions
A
highway
A
explicitly express the preferences of different drivers and
D
also
A
learned reward function is robust, which is manifested by only a marginal decline in proximity to the human driving trajectories when applying the reward function in the testing conditions. For the testing performance, the
D
significantly
A
modeling
R
by 24
(a custom metric to gauge accuracy) by 23
R
the proposed reward-function-based method delivers
these two methods deliver
D
. Moreover, it is found that estimating the response actions of surrounding vehicles plays an integral role in estimating the trajectory accurately and achieving a better generalization ability