Unifying Principals in Safe and Trusted Assistive AI  
Supervisor: Dr Yali Du 
 
AI agents are often required to assist humans in many day-to-day tasks, such as in recommendation 
systems, restaurant reservation and  self-driving cars [1]. As AI agents are  frequently evaluated in 
terms of performance measures, such as human-stated rewards, many challenges are posed. Firstly, 
due to the involvement of multiple users, agents have to learn to strike a balance between the widely 
different human preferences [3]. Secondly, while it is usually assumed that humans are acting honestly 
in specifying their preference, such as by rewards or demonstrations, the consequence of humans mis-
stating their objectives is commonly underestimated. Humans may maliciously or unintentionally mis-
state their preference, leading the assistive AI agent to perform unexpected implementations. An 
example is the Tay chatbot from Microsoft; prankster users falsify their demonstrations and train Tay
informatics-phd-projects-2022-23