A novel automated treatment planning algorithm was evaluated on 10 realistic, challenging VMAT head and neck cases in comparison with the traditional human‐driven plans used for actual treatments. Side‐by‐side comparisons were performed using objective, formalized plan quality metrics. Human‐driven plans provided more homogeneous dose distributions and the autoplans excelled at limiting the OAR doses, while still conforming to the relevant RTOG dose homogeneity requirements. The DVHs for some OARs were driven by AutoPlanning beyond the specified goals, but overall this study was not equipped to probe if the software produced Pareto‐optimal plans. AutoPlanning appears to be a robust clinical tool, but it would be, in our opinion, beneficial to improve the user controls for rebalancing the optimization compromise away from OAR sparing and towards dose homogeneity, to better fit the range of preferences of different clinicians.