Journal Article

Actor-critic multi-objective reinforcement learning for non-linear utility functions

Hayes, Conor F., Roijers, Diederik M., Reymond, Mathieu, Steckelmacher, Denis, Nowé, Ann

2023

Overview

We propose a novel multi-objective reinforcement learning algorithm that successfully learns the optimal policy even for non-linear utility functions. Non-linear utility functions pose a challenge for SOTA approaches, both in terms of learning efficiency as well as the solution concept. A key insight is that, by proposing a critic that learns a multi-variate distribution over the returns, which is then combined with accumulated rewards, we can directly optimize on the utility function, even if it is non-linear. This allows us to vastly increase the range of problems that can be solved compared to those which can be handled by single-objective methods or multi-objective methods requiring linear utility functions, yet avoiding the need to learn the full Pareto front. We demonstrate our method on multiple multi-objective benchmarks, and show that it learns effectively where baseline approaches fail.

• Publisher: Springer US,Springer Nature B.V
• Subject: Algorithms / Artificial Intelligence / Computer Science / Computer Systems Organization and Communication Networks / Decision making / Expected utility / Learning / Machine learning / Multi-Objective Decision Making (MODeM) / Multiple objective analysis / Objectives / Optimization / Preferences / Software Engineering/Programming and Operating Systems / User Interfaces and Human Computer Interaction / Utility functions