There are increasing concerns that physical material constraints threaten energy security and the growth of emerging technologies. Traditional approaches to quantify material criticality utilize single-score metrics which are narrowly focused on physical scarcity and often lead to command-and-control policies. However, a broader definition of criticality that goes beyond physical scarcity to include sustainability metrics e.g. embodied energy, political instability, economic value can provide policymakers with a more comprehensive perspective of the complex and highly interconnected relationships between indicators. We use the case of solar photovoltaic materials to demonstrate the challenges and opportunities in critical materials policy and indicator choices. For silicon-based and thin-film photovoltaics in particular, Ge, Pt, As, In, Sn and Ag were found to be the most critical relative to the 17 materials studied. Multi-metric analysis for these materials reveals tradeoffs that suggest friction between sustainable economics, political stability of supply, and environmental quality objectives.